Have you ever thought about building a Metal based 2D programming? Learning Metal is still pretty high learning curve, but what I found most difficult part is make it scrollable just like another application using UIScrollView. If you are working on 2D game programming, your users will have different user experiences and the things I write here many not be applicable, but if you working on productivity type of apps like painting and drawing, then probably you would agree what I mean here.
As you know MTKView cannot be placed in UIScrollView's subview, it just won't work. But simulating UIScrollView like experiences with gesture recognizer is pretty hard.
I recently come up with crazy idea about this. Placing MTKView behind UIScrollView and placing dummy contents view as a subview of UIScrollView. Make sure UIScrollView and dummy content view's background color to be transparent.
Ok then, when you pinch and zoom on UIScrollView, you are zooming and scrolling transparent dummy content view. Then transparent content view signals to MTKView to render content based on its dummy contents view's bounds. So, from user's eye, although MTKView is not zooming or scrolling, instead just sitting the same place behind the UIScrollView, but metal based contents actually look zooming and scrolling.
I figured out this technique for rending zoomed PDF. For more information please refer following article [in Japanese].
- UIScrollView内でPDFなどのベクター表示をうまく拡大表示する方法 http://qiita.com/codelynx/items/a2a87b053f8225782a9c
Also this sample code provides some way to blending multiple shaders much easier way. When you simply implementing multiple shaders to render a content, your code start getting messier and messier, and hard to organize render pipe line descriptors, vertex descriptors and goes on and on. But this project provides a some mechanism to split each shader related codes to separate components.
Also I tried to aim this Metal sample code to be like more Core Graphics like API. It may be still far away from the goal, but each shader related code can draw on context to build a complex visual representation on the screen.
Here are the main players. Application specific subclasses are not listed here.
- RenderView
- UIScrollView
- RenderContentView
- RenderableScene
- Renderer
- Renderable
- Shader
- RenderContext
RenderView is the main component of this architecture. MTKView, UIScrollView and other necessary views and component will be created by itself. So when if you like to use RenderView on your storyboard, just place this view and all dependent view will be created at runtime.
UIScrollView os placed as a subview of RenderView, and when you zoom and scroll operation on RenderView, this UIScrollView is the one act as scroll view. As a default, is set to minimum number of touches to scroll is 2. So, when you like to scroll, use two finger. This can be set through RenderView.
RenderContentView sits as a subview of UIScrollView and when you zoom and scroll, this is the view actually subject to be operated. Since this view is transparent, user will never see the content view. Also when you touch on the view, touch event can be deliver to forth coming RenderableScene for handling event.
RenderableScene represents an object that knows how to draw a complete content. It has its own content size, and UIScrollView behaves based on this content size. This is the place where Renderable object are kept waiting for being drawn.
Also touch events are delivered to this component, so if you like to pencil or paint like application, this is the place to handle those touch events.
Renderer is the place where Render pipeline, vertex descriptor, color sampler state exists. So, Renderer knows the format of corresponding shader's vertex format and other spec necessary to render. Same type of Renderer should be instantiated only one per device (MTLDevice).
Renderable can be an instance of Renderer. For example, a Renderer knows vertex and color format, but it doesn't know the shape or texture of the drawn object. On the other hand, Renderable knows its shape, textures and others necessary to draw on the screen.
Actual Metal shader. There are vertex shader and fragment shader program. If you like to develop your own shader, you develop Renderer, Renderable and Shader as a whole pice. So render pipe line descriptors, vertex descriptors and others specific to a shader can be managed separately. Thus, RenderableScene would just instantiate any Renderable objects and draw.
At this moment, RenderContext does not do much things. But this can be used as graphics state such as holding transform matrix, just like Core Graphics's CGContext. Furthermore, this could also hold some current color or textures, to make rendering more visually presentable.
+ RenderView + MTKView + UIScrollView + RenderContentView
Here is a quick diagram of how objects are drawn to shaders
MTKView -> RenderView -> RenderableScene -> Renderable -> Renderer -> Shader
This sample project provides three shader suites. One for colored-vertex based, second is image shaders, and third is point shaders. I believe these three shaders should be sufficient to start developing your own shader component suite.
- ColorShaders, ColorRenderer, ColorTriangleRenderable, ColorRectRenderable
- ImageShaders, ImageRenderer, ImageRenderable
- PointsShaders, PointsRenderer, PointsRenderable
As a part of experiment, this RenderView displays hybrid contents between Metal and Core Graphics. If you find a clock displaying near the center of the contents, it is actually drawn by Core Graphics. RenderableScene has a hood func draw(in context: CGContext) and all the core graphics operations made to the CGContext will be shown on the screen. So If you like to display pretty complex things such as displaying PDF contents on Metal contents, this can be done by overriding this method.
There are so many things to be redesign or refactor or enhance. Here is the short list.
Currently all three Renderers provide own class methods to prevent creating the second renderer. There must be a better way...
Currently, if all Renderer's render pipeline's colorAttachments must match or crash. Developer must keep an eye on this, but there may be a better way to do this. Probably some other pit fall I am to aware of may be exist.
- Layer support
- Refactoring Renderer or Renderable protocols
- More shader samples
Versions description:
Version 8.1 (8B62)
Apple Swift version 3.0.1 (swiftlang-800.0.58.6 clang-800.0.42.1)