INSTRUCTIONS
LECTURES
Week 6 | September 30th, 2020
Mr. Lun dropped his first lecture on Spark AR in class today. We learned the 3 main types of AR filter commonly used on Instagram: plane tracking, face tracking, and target tracking. Mr. Lun walked us through the basics of Spark AR such as understanding its interface and how the AR filter works in general. He then briefed us through each technique such as the face paintings, creating UI picker tools to change background, adding plane tracking overlays, and adding interaction patches.
Week 7 | October 7th, 2020
This week's session was consultation based as Mr. Lun opens up the chance for everyone to walkthrough their final tweaks on project 1 before its final submission. Mr. Lun also went through a detailed brief for project 2 to make sure everyone is on the right track to start their ideation process (see week 6-7 in the project progression section below for further details).
Week 8 | October 14th, 2020
No class today.
Week 9 | October 21st, 2020
As submission deadline is coming to an end, we had a check up session with Mr. Lun today regarding our progressions. Additionally, we had another lecture on creating, importing, and editing 3D objects into Spark AR using a standard 3D modelling software, Blender.
Week 10 | October 28th, 2020
It's the week of submission for the second project. Mr. Lun opened up a final consultation slots for making the final touch ups on our AR filter.
PROJECTS
Week 6 - 7 | September 30th - October 7th, 2020
A project brief was given today for the second project of the module, which was for us to create an AR filter of our own using the Spark AR software. The requirements for the projects are for us to explore the features of Spark AR and make use of them in creative ways to bring interaction alive (see figure 6.1 for the basic project requirements).
 |
| Figure 6.1. Project requirements guideline. |
As always, I commenced through the design ideation process once I fully understood the brief. Here I brainstormed through a few existing ideas that I have in mind. I wrote down all the ideas I could think of on a piece of paper and tried connecting the dots. After a few days of different idea exposure through a few references, I came up with the following draft ideas shown in figure 6.2.
 |
| Figure 6.2. Spark AR filter ideation. |
The multiple ideas then fell into one final decision after going through days of testing out Spark AR. It was necessary for me to understand the potential of the tools before digging into that final decision. The idea refinement was then made for the final filter decision, the 90's pink aesthetic.
 |
| Figure 6.3. Final filter idea, 90's pink aesthetics. |
Here I focus on creating a nostalgic, yet fantasy vibe similar to the fairy-tale scenes where things look a little worn out but with a slightly modernized era feeling. I made my mood board to support that vision of mine and to make sure I have things to refer to in future stages (see figure 6.4 for the majority of the photos I have collected).
 |
| Figure 6.4. Filter look & feel mood board |
I then analysed through the effects I have to work on and prepare a note for myself to track my working progress. These are also great reminders when working on lots of things. The following figure 6.5 shows my final form of the to-do list (it was continuous).
 |
| Figure 6.5. Continuous to-do list (final version). |
Week 8 | October 14th, 2020
As a public holiday, I spent this opportunity to experiment around and learn the functions of Spark AR. I then made my first face paint for the cheeks via Adobe Photoshop. I painted the blush using simple brush tools and collected a few assets for the freckles to refine it using the brush and stamp tools to achieve the look I wanted (figure 8.1). Layer styles were used to grab the right color to match the vibe of the filter and lighting I have in mind (figure 8.2).
 |
| Figure 8.1. Face painting for the blush and freckles on Adobe Photoshop. |
 |
| Figure 8.2. Layer style effect for the freckles. |
Once done with the main face paint, I hopped over to the drawing tablet to paint special cheek marks that are unique to each theme I'm going for: the rainbow, cloudy, and sparkly theme. The final cheek marks were shown in figure 8.4.
 |
| Figure 8.3. Drawing progress for the face paint. |
 |
| Figure 8.4. Theme cheek marks (left to right: rainbow, clouds, sparkles). |
Week 9 | October 21st, 2020
Now that the face paint assets are ready to go, I exported each of them each PNG files and went ahead to prepare my 3D assets. In this case, I opted for the demon horns look in which I've selected over the mood board from figure 6.4 earlier in the ideation phase. To my unfavourable expectations, the Spark AR Asset Library didn't have the best pairs of horns for my preference, I decided to go for individual horn set where the given 3D asset is presented in singular pairs. These provide me with the most customization options for me to craft my personalized look.
The horns were scaled and position on the head ocular mask under the face tracker, while its 3D positions are attached to the forehead patch tracker to further stabilise the position of the horns. I then made a final touch up to the look of the horns through its colors, styles, materials, and textures before moving on to the next asset.
 |
| Figure 9.1. Positioning the horns individually to the head occular. The 3D positions of both horns are attached to the forehead patch to further support its interactive movement tracking. |
This step took quite a while as I browsed through the AR Asset Library to grab a few great pairs of glasses to get that 90's look intact. For the reason that the glasses I collected share a slightly different vibe to the look, I decided to keep a few of them so the glasses options aren't limited to one. As with the horns, they are positioned in front of the head ocular object and adjusted to fit the head of the person.
The glasses were then customized to match the vibe of the pink aesthetic as seen from the mood board. Here, I located each nodes for different lens colors, frames material for each pair of glasses. Pink lens in a combo with different frame shapes and color gives it the final look. Figure 9.2 shows a comparison of the final version of the glasses to its original form.
 |
| Figure 9.2. Comparison of the original and the final touch up on the 3D glasses. |
I then browsed through for the backgrounds of the filter, and went through several trials and errors with the following wallpaper shown in figure 9.3.
 |
| Figure 9.3. Aesthetic wallpapers. |
Given a fair time of judgement, I proceeded to pick its final look with 3 themes I have in mind, the rainbow, the sparkly glitters, and the cloudy theme. Both are prepared in its original wallpaper form and its icons for the UI picker.
 |
| Figure 9.4. Final background and icon theme set. |
The backgrounds were then inserted to the scene using different rectangles and the alpha from the person segmentation mask. Each wallpaper were then separated into the UI Picker patch where its visibility will only activate upon its selected UI options. The icons were then placed into each texture slots matching its wallpaper.
 |
| Figure 9.5. Adding background and icons to the filter using the Picker UI interaction patch. |
Week 10 | October 21st, 2020
Its finally the time to compile everything up and finish off the project with interaction patches. I started off with the sparkle effects using the customized simple kira patch. These patches were then connected to the texture patch of the sparkle files (I had 3 of them) and the adjustments of the sizes, shifts and spawn rate followed. To enable the display, the camera texture and device resolution is attached to the input of the simple Kira patch (see figure 10.1).
 |
| Figure 10.1. Sparkle effects interaction patch. |
Following the sparkle effects, I prepared the color LUT files for the filter effect of the foreground. This is done by connecting the FastColorLUT patch to the texture patch of rectangles in which I have applied each color LUT to. The display is again enabled by the camera texture and LUT slots in which I have collected. These color LUT are prepared separately for each themes I've set in the earlier phase.
 |
| Figure 10.2. Color LUT interaction patch. |
Its never a bad thing to have a thorough checkup on the plenty of effects, assets, and patches I have added in this project so far. To make sure things turn out right, I tested out the filter on different faces, including myself as given below in figure 10.3.
 |
| Figure 10.3. Filter effects test and check up on different faces. |
Moving on, I prepared some foreground elements to emulate the VHS recorder effect on Adobe Illustrator. As I intended to make the red circle blink like a recorder, I compiled them into an animation file of 2 frames (one being invisible) and enabled the loop function to leave the blinking on as long as the camera shutter is not pressed. The process and the settings are shown below in figure 10.4.
 |
| Figure 10.4. Assets, process, and settings to create the blinking circle. |
The VHS rgb screen shifting effect was then introduced to the filter using a customized RGB Shifter patch from the AR Library. The process is applied to the filter using a similar method to the sparkle effect where the camera texture was used to host the interaction patch whilst connecting it to a rectangle with the rgb shifter effect applied on.
 |
| Figure 10.5. Rgb shifter effect (isolated). |
 |
| Figure 10.6. Testing out the effect. |
To give the filter a much more versatile control to the user, I gave each assets a boolean control function for different users to play around with and make the best out of the options the filter has to offer.
As the horns may not appeal to every user, the user can simply press on the forehead area to enable/disable the horn. This is done through an invisible plane object, in which was located and scaled to appropriate size right on the forehead area.
Moreover, the glasses followed the same idea where users can tap on the tap to switch through each glasses and they can even disable it completely if they wished to. The process was almost the same to the horn as the invisible plane was also made as a switch for the command. However, with more than just true/false boolean values, a counter was necessary for the system to count the amount of times the function was executed. The amount of tap was counted by the equal exactly patch in which will execute the visibility of each objects I assigned to according to the numbers I've set for each patch (see figure 10.7 for the compiled patch of the horns and glasses interaction patch).
 |
| Figure 10.7. Plane tap interaction patch: Tap forehead to change horn's visibility (top) - Tap face to equip different glasses (bottom). |
Next up, for the special cheek marks I arranged for each theme (see figure 8.4), I applied a face interaction for the effect to provide users the versatility to further empower their control over the filter. Put it simply for the interaction patch, the action of opening mouth will act as a switch for turning the cheek marks on or off using the pulse-switch patch to activate it.
 |
| Figure 10.8. Open mouth to apply theme cheek mark effects (Effects are combined in one interaction). |
I then decided to give each theme a unique look every time it changes by changing its color LUT, which is essentially a color filter for the screen, and the 3 cheek marks I set earlier. Choosing a cloudy theme shows up a cloudy background, cloudy cheeks, and color vibe that fits with the background theme, while choosing other themes does the same magic.
 |
| Figure 10.9. Filter theme effects interaction patch: background, cheeks, color LUT. |
Mobile walkthrough video:
Comments
Post a Comment