iOS 27: Media Intelligence - by Anton Gubarenko
Anton’s Substack
SubscribeSign in
iOS 27: Media Intelligence Framework<br>New framework to analyze video content and group faces
Anton Gubarenko<br>Aug 03, 2026
Share
Apple added a new framework in iOS 27 (well silently added in Beta section), and at first glance it might look like another layer on top of Vision. It is already have a lot of capabilities, but the interesting part starts when we stop thinking about one image and where is it on photo.<br>Media Intelligence works with a collection. It can keep face analysis between launches, group appearances of the same person across different photos, and find useful moments inside a video. Vision gives us observations. Media Intelligence starts connecting them.<br>Sounds useful, especially for photo-heavy apps. But before we start:<br>This feature is still in beta and might change before the final release. Warning you as usual.
What It Does
Media Intelligence currently gives us two main analyzers:<br>FaceGroupAnalyzer
VideoAnalyzer
The names are quite straightforward.<br>FaceGroupAnalyzer detects faces, stores the results, and tries to group appearances that belong to the same person.<br>VideoAnalyzer works with requests such as KeyFrameAnalysisRequest and HighlightAnalysisRequest to find representative or interesting moments in a video.<br>Vision is not going anywhere. It still handles OCR, landmarks, poses, masks, and frame-by-frame analysis. Media Intelligence simply sits one level above and owns a few workflows that otherwise require persistence, matching, and a lot of glue code.<br>Analysis happens on-device. Face data does not need to be uploaded to a server, and the analyzer can reuse its local index between launches. Nice for privacy, but not a free pass. A generated group is still not a verified identity.<br>So explain why the app analyzes people, keep user-assigned names outside the analyzer’s working directory, and give users a way to remove generated data.
A Real Device Is Required
The sample targets iOS 27 and exposes only MediaIntelligenceView, so you can present it from an existing app without creating a separate Scene.<br>And here is the first practical surprise: a supported physical device is required . Run the analyzer in Simulator and you can get a very descriptive “Can’t create context” error. Thank you, beta SDK.<br>Simulator can still render the surrounding SwiftUI interface, but it does not reproduce the hardware-backed Media Intelligence pipeline. The sample therefore shows ContentUnavailableView when compiled for Simulator:<br>#if targetEnvironment(simulator)<br>ContentUnavailableView(<br>"Physical Device Required",<br>systemImage: "iphone.gen3.slash",<br>description: Text(<br>"Media Intelligence analysis must be tested on a supported physical device."<br>#else<br>// iOS 27 content<br>#endifThere is also no public generic Boolean such as isNeuralEngineAvailable. On a physical device, analyzer initialization or processing can still fail, so the app should treat those errors as a real unavailable state rather than showing an endless spinner.<br>Now we can finally create the analyzer.
FaceGroupAnalyzer
FaceGroupAnalyzer looks like one type, but it actually handles four different jobs:<br>ingest image assets;
detect faces;
persist detections;
cluster detections into people.
Face detection itself is not the most difficult part here. Keeping the results stable and grouping them later is where this API becomes interesting. Whole implementation will be provided below. It requires some help-structs so we will focus on main points.<br>Creating the Analyzer
The analyzer needs a writable working directory. I use Application Support here because this is framework-managed persistent data, not a temporary cache:<br>let workingDirectory = URL.applicationSupportDirectory<br>.appending(path: "FaceGroupData", directoryHint: .isDirectory)
try FileManager.default.createDirectory(<br>at: workingDirectory,<br>withIntermediateDirectories: true
let analyzer = try FaceGroupAnalyzer(<br>workingDirectory: workingDirectory<br>)Reuse the same directory between launches. Otherwise you lose the main benefit of the framework and start rebuilding the index again and again.<br>Also, do not put your own files there or try to “fix” its contents manually. The directory belongs to the analyzer.<br>Stable Asset IDs
The framework gives us a dedicated MediaIntelligenceImageAsset.ID, so I keep it strongly typed instead of converting everything to String too early:<br>struct Photo: Identifiable, Hashable {<br>let id: MediaIntelligenceImageAsset.ID<br>let name: String<br>let url: URL<br>let image: UIImage<br>}Assets are then created with the same stable ID:<br>let assets = preparedPhotos.map { photo in<br>MediaIntelligenceImageAsset(<br>id: photo.id,<br>kind: .url(photo.url)<br>}This ID is important. insertOrUpdateAssets(_:) uses it to understand whether the image is new or whether we are updating the same logical asset. Random IDs here would quietly turn every run into another import.<br>Preparing the Images
For the demo, I use regular...