Zhuowei Zhang on Nostr: I think I figured out where the Vision Pro stores fisheye distortion data for playing ...
I think I figured out where the Vision Pro stores fisheye distortion data for playing back Apple TV Immersive videos.:
It’s in /System/Library/PrivateFrameworks/CoreIMS.framework and selected by the video using embedded JSON.
I read nprofile1qy2hwumn8ghj7un9d3shjtnddaehgu3wwp6kyqpqcln0h9yfusmptwdaaxmcfeg9yy9lcgths77dk4jw2dgjfc6u6d0qh83eqg (nprofile…3eqg) ‘s blog post about Apple TV Immersive videos’ distortion, so I decided to grab the Apple TV logo from an immersive video (Immersed) and check for myself.
I opened the Apple TV logo from the Immersive video (the same file the blog post investigated).
Oddly enough, I didn’t see the 45-degree rotated video - instead, the Apple TV logo was undistorted.
I opened the video in a hex editor and saw some blobs of JSON - probably part of the mebx timed metadata track.
[{"id":1,"command":"setCamera","time":0,"args":{"cameraId":"_default_preroll_rect_lens_01"}}]
grepping for _default_preroll_rect_lens in the visionOS Simulator finds a match in /System/Library/PrivateFrameworks/CoreIMS.framework/CoreIMS, along with other files that refer to this name:
CoreIMS
Info.plist
_CodeSignature
_default_preroll_cg_lens_01.png
_default_preroll_cg_s45_01.png
_default_preroll_cg_s45_01.usdz
_default_preroll_rect_lens_01.png
default.metallib
nomask.png
It looks like there’s two/three? lens:
_default_preroll_rect_lens_01 - what this logo uses,
_default_preroll_cg_lens_01
and _default_preroll_cg_s45_01.
Based on the name, that last lens sounds like a rotated 45 degree lens? Probably that’s the version of the logo that anyware looked at.
The _default_preroll_rect_lens_01.png seems to be the eye mask for each lens (the fade out area around each eye)
The _default_preroll_cg_s45_01.usdz is a half-sphere that looks like the projection mesh.
And since this is on a timed metadata track, the video can probably switch to different projections at any time?
CoreIMS also mentions a “Venue” data, with keys like leftControlPoints / rightControlPoints, so it sounds like videos can define their own distortion?
https://mastodon.social/@anyware/112272341337884367
It’s in /System/Library/PrivateFrameworks/CoreIMS.framework and selected by the video using embedded JSON.
I read nprofile1qy2hwumn8ghj7un9d3shjtnddaehgu3wwp6kyqpqcln0h9yfusmptwdaaxmcfeg9yy9lcgths77dk4jw2dgjfc6u6d0qh83eqg (nprofile…3eqg) ‘s blog post about Apple TV Immersive videos’ distortion, so I decided to grab the Apple TV logo from an immersive video (Immersed) and check for myself.
I opened the Apple TV logo from the Immersive video (the same file the blog post investigated).
Oddly enough, I didn’t see the 45-degree rotated video - instead, the Apple TV logo was undistorted.
I opened the video in a hex editor and saw some blobs of JSON - probably part of the mebx timed metadata track.
[{"id":1,"command":"setCamera","time":0,"args":{"cameraId":"_default_preroll_rect_lens_01"}}]
grepping for _default_preroll_rect_lens in the visionOS Simulator finds a match in /System/Library/PrivateFrameworks/CoreIMS.framework/CoreIMS, along with other files that refer to this name:
CoreIMS
Info.plist
_CodeSignature
_default_preroll_cg_lens_01.png
_default_preroll_cg_s45_01.png
_default_preroll_cg_s45_01.usdz
_default_preroll_rect_lens_01.png
default.metallib
nomask.png
It looks like there’s two/three? lens:
_default_preroll_rect_lens_01 - what this logo uses,
_default_preroll_cg_lens_01
and _default_preroll_cg_s45_01.
Based on the name, that last lens sounds like a rotated 45 degree lens? Probably that’s the version of the logo that anyware looked at.
The _default_preroll_rect_lens_01.png seems to be the eye mask for each lens (the fade out area around each eye)
The _default_preroll_cg_s45_01.usdz is a half-sphere that looks like the projection mesh.
And since this is on a timed metadata track, the video can probably switch to different projections at any time?
CoreIMS also mentions a “Venue” data, with keys like leftControlPoints / rightControlPoints, so it sounds like videos can define their own distortion?
https://mastodon.social/@anyware/112272341337884367