To hide the limited Field of View of a Mixed Reality headset like the HoloLens or the Magic Leap you can fade out the holograms at the border of the view. I will discuss three possible techniques with different advantages and disadvantages.
Note that all techniques have the same visual result in the HoloLens. However when recorded with Mixed Reality Capture the border effect seems to largely fall outside of the MRC camera field of view.
Postprocessing effect
The most modern solution is applying a post-processing effect. Post-processing effects in Unity can be a heavy hit on fillrate so that is why Microsoft advises not to use them on HoloLens. The Magic Leap has a bit more graphics power to spend so it may be a viable solution on that device. Typically a post-processing effect works by rendering the scene to a texture and then rerender that texture on a screen-aligned quad with a filter effect (e.g. grayscale, bloom, vignette etc.) applied during that rerender. This can be done multiple times in succession, but since each rerender also means modifying each screen pixel this will cost you fillrate.
Post-processing a scene in Unity needs two assets to work together:
- a script that is attached to the camera and implements OnRenderImage
- a shader that is applied when we rerender the scene image in OnRenderImage
The biggest advantage of using this technique is that it doesn’t require changes to the content of your scene. But it may be a bit overkill for only add a fading border.
Postprocessing shader with red/green output
Postprocessing final result
BorderFadePostProcess.cs
using UnityEngine;
public class BorderFadePostProcess : MonoBehaviour
{
[Range(0, 500)]
public float borderWidth = 100;
private Material material;
void Awake()
{
// Creat a Material using the FadeBorder shader
material = new Material(Shader.Find("Hidden/FadeBorder"));
}
// Postprocess the image
void OnRenderImage(RenderTexture source, RenderTexture destination)
{
material.SetFloat("_borderWidth", borderWidth);
// Rerender the scene on a screen aligned quad with the given material/shader
Graphics.Blit(source, destination, material);
}
}
Fadeborder.shader
Shader "Hidden/FadeBorder"
{
Properties
{
_MainTex ("Texture", 2D) = "white" {}
}
SubShader
{
// No culling or depth
Cull Off ZWrite Off ZTest Always
Pass
{
CGPROGRAM
#pragma vertex vert
#pragma fragment frag
#include "UnityCG.cginc"
struct appdata
{
float4 vertex : POSITION;
float2 uv : TEXCOORD0;
};
struct v2f
{
float2 uv : TEXCOORD0;
float4 vertex : SV_POSITION;
};
v2f vert (appdata v)
{
v2f o;
o.vertex = UnityObjectToClipPos(v.vertex);
o.uv = v.uv;
return o;
}
sampler2D _MainTex;
// Width of the border in pixels
fixed _borderWidth;
float linearStep(float a, float b, float t)
{
return saturate((t - a) / (b - a));
}
fixed4 frag (v2f i) : SV_Target
{
fixed4 col = tex2D(_MainTex, i.uv);
// Distance to border along x and y
float2 distanceInPixels = (0.5 - abs(i.uv.xy - 0.5)) * _ScreenParams.xy;
// Linear border fade
float mask = linearStep(0, _borderWidth, min(distanceInPixels.x, distanceInPixels.y));
//return col + lerp(fixed4(1, 0, 0, 1), fixed4(0, 1, 0, 1), mask);
// Return masked color
return col * mask;
}
ENDCG
}
}
}
Fading materials
Another solution is to let the material/shader do the fading out. The used shader needs to calculate the screen position and then fade out the material when at the edge of the screen. Each hologram in the scene needs to have a material applied with this edge fading in it’s shader.
Modifying the used shaders may result in an improved performance, but this highly depends on what is visible in your scene. No separate postprocessing render is needed, but each shader uses a few more instructions and it requires all shaders in your scene to be modified.
Material fade with red/green output
Material fade final result
See GitHub project link for the source code of the Standard_BorderFade shader. It’s a modification of the Standard shader from the Holotoolkit.
Screenspace border
The third solution is rendering a screen aligned border that fades from transparent block to opaque black. It may sound a bit old skool, but it does the job with minimal performance impact and no need to modify the content of your scene.
Screenspace border with red/green output
Screenspace border final result
Borderdrawer.cs
using UnityEngine;
public class BorderDrawer : MonoBehaviour
{
[Range(0, 500)]
public float borderWidth = 100;
private Material material;
private Color OutsideColor = new Color(0, 0, 0, 1);
private Color InsideColor = new Color(0, 0, 0, 0);
private void Start()
{
// Creat a Material using the FadeBorder shader
material = new Material(Shader.Find("Unlit/VertexColor"));
}
void OnPostRender()
{
// HoloLens resolution 1280 x 720
// HoloLens screen/safe area 1268 x 720
float left = (float)borderWidth / (float)Screen.width;
float bottom = (float)borderWidth / (float)Screen.height;
float right = 1 - left;
float top = 1 - bottom;
GL.PushMatrix();
material.SetPass(0);
GL.LoadOrtho();
GL.Begin(GL.TRIANGLE_STRIP);
GL.Color(OutsideColor);
GL.Vertex3(0.0F, 0.0F, 0);
GL.Color(InsideColor);
GL.Vertex3(left, bottom, 0);
GL.Color(OutsideColor);
GL.Vertex3(1F, 0F, 0);
GL.Color(InsideColor);
GL.Vertex3(right, bottom, 0);
GL.Color(OutsideColor);
GL.Vertex3(1F, 1F, 0);
GL.Color(InsideColor);
GL.Vertex3(right, top, 0);
GL.Color(OutsideColor);
GL.Vertex3(0F, 1F, 0);
GL.Color(InsideColor);
GL.Vertex3(left, top, 0);
GL.Color(OutsideColor);
GL.Vertex3(0F, 0F, 0);
GL.Color(InsideColor);
GL.Vertex3(left, bottom, 0);
GL.End();
GL.PopMatrix();
}
}
Github
The source code for these three techniques is available from this Github repository
References
- Astronaut model on Remix3D (no longer available)
- Performance recommendations for HoloLens apps
Leave a Reply
You must be logged in to post a comment.