Information AboutCore Image |
| CATEGORIES ABOUT CORE IMAGE | |
| mac os x apis | |
|
Core Image is a new technology in Mac OS X that heavily leverages the machine's GPU for image related effects. Demonstrated in the August 2004 WWDC , it provides access to the most common graphics filters. It adds the ability to do many common types of Image Processing On-the-fly (which previously required custom implementations in the developers' own code). Image units can be tested with the Core Image Fun House application. Core Image has further implications than simple generated by Dashboard . Developers can write their own filters, Photoshop filters, and package them in bundles called Image Units. However, unlike a traditional filter that manipulates the source image, Core Image performs a manipulation as an overlay — maintaining the original image. This is called "non-destructive" manipulation. The result is the ability to do highly complex and layered image manipulations with little or no loss in quality. Transition effects work like filters, combining two or more images to form a result image, and accepting a time value (0 < t < 1) as a parameter. By driving an animation using a timer, a transition from one image to another is easily implemented. The components of pixels in Core Image are represented using Floating Point values within the designated Color Space . The use of floating point allows great precision. Multiple effects can be achieved by combining several Image Units into a sequence called an effects stack. However, for most Image Units, Core Image does not generate each intermediate image as it processes the stack; instead, it calculates the resulting mathematical operation once, then applies that result to the image. The upshot of this is that performance does not degrade dramatically as more effects are added - the pixels are only processed once no matter how complex the effects stack becomes. Core Image is designed to leverage the processing abilities of certain types of Graphics Processor found on high-end video cards. This gives enormous performance benefits because large image data sets need not be moved across the main Memory Bus , which is one place where real-time image processing can become Bandwidth bound. Core Image, being only for two-dimensional graphics, uses only the Pixel Shader s of the GPU, not the vertex shaders. This means that the more pixel shader pipelines a GPU has, the better the performance will be. Other considerations are VRAM bandwidth, memory bus width, and GPU clock speed. However, if a suitable graphics processor is not available, Core Image will fall back to the next best option: using the AltiVec vector-processing abilities of the G5 and G4 processors, or plain scalar code on a G3 . Core Image will automatically adopt the fastest approach for a given machine architecture. However, realistically, real-time processing requires a compatible GPU. When the AltiVec-fallback is used (as recommended by Apple for final rendering, most likely because VRAM readback over AGP is very slow), Core Image performance scales almost linearly with the number of CPUs and their clock speed. CORE VIDEO Running alongside Core Image and built upon it is Core Video . In the same spirit as that of Core Image, it allows on-the-fly editing of video, as shown in the WWDC Webcast. Both technologies depend heavily on the system's Graphics Card . COMPATIBLE GPUS Basically any Mac-compatible GPU that supports at least second-generation shaders, more specifically the ARB_fragment_program OpenGL extension, can also do Core Image. The same requirements are used by Quartz Extreme . Namely, the following GPUs are supported (as of March 2006): ATI
Intel
NVIDIA
Note: The Geforce FX 5200 is only barely suited for Core Image due to poor pixel shader performance. With most Macs, the CPU is faster than the 5200 in rendering, so Apple disabled Core Image support for it and also did not support the 5200 in Aperture . APPLE APPLICATIONS THAT USE CORE IMAGE Core Image is part of the System and can be implemented in any application. The AltiVec- and PPC-fallback approach ensures compatibility on any system. It was introduced with Mac OS X V10.4 "Tiger", so prior versions don't support it. Apple ships a program called Quartz Composer with its Developer Tools. Its maker is Pierre-Olivier Latour, who is now employed at Apple. Quartz Composer is basically Version 2 of his program Pixelshox, which is not supported any more, but which also runs on earlier versions of MacOS X. Both Pixelshox and Quartz Composer support an approach similar to Apples compositing-program Shake in which you can generate OpenGL-based scenes by linking various building blocks with nodes, feeding the output from one module to another. A multitude of data types (e.g. Boolean, numeric, image, video, RGB) and inputs (e.g. DV, image, video, generators, mouse, RSS-feed) are supported, and in- and outputs can be mapped to programs in Xcode. People without any programming knowledge can build complex scenes by just linking various blocks in the right way. Unlike Pixelshox, Quartz Composer now also supports Core Image Filters (delimited by a dot infront of the Module). Since Quartz Composer allows to export the scene as a QuickTime 7 movie, developers can embed these, and thus be able to use Core Image in web pages. Since Quartz Composer only exports the scene and its assets to QuickTime and not rendered frames of video, the resulting file is very small. This means, though, that the movie requires Core Image, so it will not work on Windows, nor on versions of Mac OS X prior to Mac OS X 10.4, even with QuickTime 7 installed. Also in the Developer Tools is Core Image Fun House, similar to another Core Image-based program called iMaginator. Outside of the Developer Tools, Dashboard uses Core Image for the Water Ripple effect, Apple's photography software Aperture uses it for other things, and some of IMovie 's filters use Core Video. Apple's title generator Motion uses similar GPU-based-rendering techniques, but does not (yet) use Core Image, using its own frameworks instead. This is done for Mac OS X V10.3 compatibility. IMAGE UNITS
EXTERNAL LINKS
|
|
|