================== 3D Community Mural ================== This project includes the server side component to the 3D Community Mural project. The 3D aspects come from the use of OSPRay on the server side, following a Tapestry-like architecture. How to Run ---------- Before you run these commands, you need to have OSPRay downloaded. You can download it from its Releases page on GitHub [0]. We are using version 1.8.5. To run the server, we need to create a Docker image with OSPRay installed. You can do this by first running: $ ./go.sh build Then to actually run the server: $ ./go.sh server Then open http://localhost:8801 and you will see the scene rendered. You can click and drag on the image that's returned to rotate around the scene. Zooming does not currently work well. Internally, the "./go.sh server" command passes several arguments to the server code automatically. For instance, it passes the location of the materials.txt file. [0]: https://github.com/ospray/ospray/releases Using the Service ----------------- This service is best described as a web-accessible rendering service that can be used entirely from requesting special URLs and getting images (JPEGs) back. This means that after constructing a URL you can use it in an HTML image tag. The Seelab team will handle running the service until the MVP is finished, after which point, we will hand off a Docker image that can be run on production systems. The Seelab base URL is: http://accona.eecs.utk.edu:8801/ The service is entirely stateless, so it will not remember anything between requests. This means that everything it needs to know to render the scene will have to be passed every request. This means we will need to know the camera position, camera view direction, camera up direction, desired resolution, and any scene information. For the community mural, most of these parameters can be fixed, which simplifies the requests. Here is a pair of JavaScript functions. The first one requires all information in the Tapestry URLs. The second one has a simplified interface that only needs the camera horizontal/vertical positions, image resolution, and ball position. function makeTapestryURL(options) { const O = options; const parts = [ O.prefix, // base URL 'image', O.scene, O.px, O.py, O.pz, // position O.ux, O.uy, O.uz, // up O.vx, O.vy, O.vz, // view O.quality, // resolution (square image) O.extra, // extra parameters, like tiling ]; return parts.join('/'); } function makeMuralObject(options) { const O = options; const parts = [ 'obj' + O.number, // e.g. "obj1" O.object, // what object/model to use O.bx, O.by, O.bz, // object position O.bsx, O.bsy, O.bsz, // object scale O.brx, O.bry, O.brz, // object rotation (degrees) O.material, // ball material ]; return parts.join(','); }; function makeMuralWall(options) { const O = options; const parts = [ O.name, O.r, O.g, O.b, ]; return parts.join(','); }; function makeMuralLight(options) { const O = options; const parts = [ 'light', O.x, O.y, O.z, O.r, O.g, O.b, O.intensity, ]; return parts.join(','); }; function makeMuralURL(options) { const O = options; const lightParts = O.lights.map((light) => { return makeMuralLight(light); }); const sceneParts = [ 'scene', makeMuralObject(O.obj1), makeMuralObject(O.obj2), makeMuralObject(O.obj3), makeMuralWall(O.negx), makeMuralWall(O.posx), makeMuralWall(O.negy), makeMuralWall(O.posy), makeMuralWall(O.negz), makeMuralWall(O.posz), lightParts.join(','), ]; const extraParts = []; if (O.bgcolor) extraParts.push('bgcolor', bgcolor.join(',')); return makeTapestryURL({ prefix: 'http://accona.eecs.utk.edu:8801', scene: sceneParts.join(','), px: O.cameraHorizontal, // camera x py: O.cameraVertical, // camera y pz: -2.0, // camera depth ux: 0, uy: 1, uz: 0, vx: 0, vy: 0, vz: 1, quality: O.quality, extra: extraParts.join(','), }); } For example, you could use this to construct a render looking directly at a ball, that is directly in the center of the scene. const url = makeMuralURL({ cameraHorizontal: 0.0, cameraVertical: 0.0, quality: 512, obj1: { number: 1, object: 'Deer', bx: 0.0, by: 0.0, bz: 0.5, bsx: 1.0, bsy: 2.0, bsz: 1.0, brx: 0.0, bry: 0.0, brz: 120.0, material: 'Emerald', }, obj2: { number: 2, object: 'Angel', bx: 0.5, by: 0.0, bz: 0.5, bsx: 1.0, bsy: 2.0, bsz: 1.0, brx: 0.0, bry: 0.0, brz: 0.0, material: 'Emerald', }, obj3: { number: 3, object: 'Debris', bx: 0.0, by: 0.5, bz: 0.5, bsx: 1.0, bsy: 2.0, bsz: 1.0, brx: 0.0, bry: 0.0, brz: 0.0, material: 'Brass', }, negx: { name: 'negx', r: 0.8, g: 0.2, b: 0.2, }, posx: { name: 'posx', r: 0.2, g: 0.8, b: 0.2, }, negy: { name: 'negy', r: 0.2, g: 0.8, b: 0.8, }, posy: { name: 'posy', r: 0.8, g: 0.8, b: 0.8, }, negz: { name: 'negz', r: 0.8, g: 0.2, b: 0.8, }, posz: { name: 'posz', r: 0.2, g: 0.2, b: 0.8, }, lights: [ { x: 0.0, y: 0.9, z: 0.5, r: 0.8, g: 0.1, b: 0.1, intensity: 10.0, }, ], bgcolor: [255, 0, 0, 255], }); This results in: http://accona.eecs.utk.edu:8801/image/scene,obj1,Deer,0,0,0.5,1,2,1,0.0,0.0,120.0,Emerald,obj2,Angel,0.5,0,0.5,1,2,1,0,0,0,Emerald,obj3,Debris,0,0.5,0.5,1,2,1,0,0,0,Brass,negx,0.8,0.2,0.2,posx,0.2,0.8,0.2,negy,0.2,0.8,0.8,posy,0.8,0.8,0.8,negz,0.8,0.2,0.8,posz,0.2,0.2,0.8,light,0.0,0.9,0.5,0.8,0.1,0.1,10.0,0.0/0/0/-2/0/1/0/0/0/1/512/bgcolor,255-0-0-255 The scene has a fixed size and has limits on X, Y, and Z. Anything (camera or ball) can be put anywhere in this fixed space, but going outside of it (e.g. X=100) will not work as expected. X: -3.45 (left) to +3.45 (right) Y: -3.45 (bottom) to +3.45 (top) Z: -9 (far end of the box) to +1 (far end of the box) Creating OBJ Models ------------------- Note: We currently do not use this conversion process. It could be added back, but now that we are using OSPApp as the backend for our OSPRay process, we get OBJ parsing for free. Materials --------- There is a set of preloaded materials available to the user of the service. These are based on a list available at: http://web.eecs.utk.edu/~huangj/cs456/materials_ogl.htm They are referenced by name, with internal spaces removed. The current master list of names is: Brass Copper Pewter Jade Turquoise Bronze PolishedCopper Silver Obsidian BlackPlastic PolishedBronze Gold PolishedSilver Pearl BlackRubber Chrome PolishedGold Emerald Ruby Internally, these materials are loaded into the OSPRay process via environment variables. On variable tells the number of materials to load, and then the remaining variables follow a specific naming scheme. nmat: Number of materials to load (e.g. 12). mat_0: First material. mat_1: Second material. ... mat_11: Last material. Each of the mat_ variables is a whitespace-delimited set of floats (prefixed with a name). For example, Brass' environment variable value is: Brass 0.329412 0.223529 0.027451 1.0 0.780392 0.568627 0.113725 1.0 0.992157 0.941176 0.807843 1.0 27.8974 These fields are, in order: Name, Diffuse R, G, B, Opacity, Specular R, G, B, Shininess The server loads these from a materials.txt in the root of this project. To create your own, add a line like the Brass one above to the end of materials.txt. For instance, an opaque red matte material might look like: Red 0.8 0 0 1 0.4 0 0 10 Then when you reload the server, you can refer to it by the name "Red". Models ------ There is a list of preloaded OBJ models. They are referenced by name with internal spaces removed. The current list of models is: Angel Flower Home_08 Arm Hat Home_09 Banana HighRise_01 Horse BlueWhale HighRise_02 Hospital_02 Building_01 HighRise_03 Hospital Building_02 Home_010 Island Building_03 Home_011 PoloPants Cactus Home_01 Restaurant_01 Church Home_02 Shop_1 Crystal Home_03 Stadium Debris Home_04 Umbrella Deer Home_05_v2 midSizeBuildingComplex DinoSkull Home_06 wooden_door EnergyPlant Home_07 Note: Currently the objs.txt file is not used for the list of preloaded OBJ files. Instead, the scene.sh script exists to preload all of the objects via the command line. This also means that converting OBJ files to binary files is not needed. From the technical side what happens is: when an object is created from the command line, it gets added to OSPRay's scene with a name like: translate_Actual_Object_Name_0_0_0 Tapestry loops through all objects in the scene and extracts the Actual_Object_Name from the name OSPRay gives it. This Actual_Object_Name acts as the identifier for obj1, obj2, etc controls. Walls ----- Note: This currently does not work due to how we are loading the scene. To make this happen again, we would need to add a special OBJ file for each wall and set the material for each one. You can change the color of each wall to any RGB color. Each wall is labeled as {Neg,Pos}{X,Y,Z}. In the scene URL, you would give a particular wall name and then 3 floats. The exact wall names are: negx negy negz posx posy posz An example specification for making the negx wall red is: negx,0.8,0.2,0.2 Lights ------ There are currently 6 available light sources, all of them point lights. Their position, color, intensity, and radius can be changed. In the URL, each light source is indicated by a bare "light" name, as opposed to objects that are numerically suffixed. This means that if you want multiple lights, it looks like: light,(params for light 1),light,(params for light 2) Unused lights will be reset to a default position so if you specify 3 lights the first time and 2 the second, you'll only see 2 lights. Fewer than 3 Objects -------------------- The renderer is hard coded to use 3 objects. If you want to use fewer than 3, then the best course of action is to position the unwanted objects outside of the box. For instance, setting X to 100 should get it far enough away to not affect anything. All of the other object parameters must be set correctly though or else the request will fail. So one way to do this is to keep a base object specification and then use then when needed: const baseObject = { object: 'Donut', bx: 100.0, by: 100.0, bz: 100.0, bsx: 1.0, bsy: 2.0, bsz: 1.0, material: 'Emerald', }; const obj1 = Object.assign({}, baseObject, { number: 1, }); Background Color ---------------- Some renders do come back with transparent sections. Because we want fully opaque images, we need to substitute a background color for all transparent sections. To fascilitate this, there's now a control to change this background color. This goes in the "extra" section at the end of the URL and looks like: http://hostname:port/scene,.../X/Y/Z/UX/UY/UZ/VX/VY/VZ/SIZE/bgcolor,R-G-B-A Here R, G, B, and A are integers from 0 to 255. For instance, solid white (the default if you don't pass anything) is: bgcolor,255-255-255-255 Solid red would be: bgcolor,255-0-0-255 And so on. Miscellaneous Notes ------------------- In the OSPRay code, it would be nice to be able create an instance with a different material than the instanced geometry. However, based on Instance.ispc in the OSPRay library, it only uses the instanced geometry's material. https://github.com/ospray/ospray/blob/43a89186b4dc8a17dc130e3295ad96ddf69cb029/ospray/geometry/Instance.ispc#L41-L43 OSPRay never explicitly defines what an affine3f is. You can find a struct definition in the ospray.h header file, but all you get is: typedef struct { osp_vec3f vx, vy, vz; } osp_linear3f; typedef struct { osp_linear3f l; osp_vec3f p; } osp_affine3f; To get the real definition, you have to look at Instance.h: Once created, a trianglemesh recognizes the following parameters <pre> float3 "xfm.l.vx" // 1st column of the affine transformation matrix float3 "xfm.l.vy" // 1st column of the affine transformation matrix float3 "xfm.l.vz" // 1st column of the affine transformation matrix float3 "xfm.p" // 4th column (translation) of the affine transformation matrix OSPModel "model" // model we're instancing (Side note: I think "trianglemesh" above should be "instance"). A question I had is: why not just store the whole matrix? For affine transformations, you generally don't (never?) change the last row and leave it as 0, 0, 0, 1. In other words, for most use cases, your matrix will look like: l.vx l.vy l.vz p [ [ XSCALE, 0, 0, XTRANS ] [ 0, YSCALE, 0, YTRANS ] [ 0, 0, ZSCALE, ZTRANS ] [ 0, 0, 0, 1 ] ]