[Postmortem] Developing an educational, kiosk, VR game for Windows Mixed Reality

In this article I analyze the development Voedingscentrum VR, an educational, kiosk-style and Virtual Reality game I helped to develop at Fantazm from October 2018 until April 2019. After a quick introduction about the project, the biggest problems that surfaced during development are explained along with the solutions we found to solve them. At the end, a quick conclusion wraps the article up.

Introduction

The project was commissioned by Voedingscentrum (The Netherlands Nutrition Center), an independent organization funded by the Dutch government. Their aim is to use relevant research regarding sustainable nutrition to promote balanced, sustainable food consumption and production.

The concept

The game was planned as a fun, educacional VR experience targeted at children from 9 to 12 years old. Players are placed into a cartoonish world where they need to feed Smikkel – an elephant-like creature – for 3 to 4 meals: breakfast, lunch, afternoon snack and dinner.

smikkel

Smikkel eating a sandwich from a plate

A robot waiter teaches the players about the game instructions and loop: for each meal the player chooses, Smikkel comes to the dinner table and you feed him by choosing, grabbing and placing food items (served by the waiter) on a plate. Once all the food items of a specific meal are chosen (between 2 and 6 items), Smikkel eats the food and its mood, energy levels and appearance change according to the food items you fed him. Following, playtime starts and the player can interact with Smikkel by petting it and throwing balls so it can fetch them. During playtime, it should be evident that the food Smikkel ate affected its behavior. Additionally, the waiter displays a report showing the selected food items and gives the user feedback on the chosen food.

waiter

The waiter

Once playtime finishes, the feeding loop starts again. Once all the meal loops are over, the waiter gives a final feedback, announces that the experience is over and instructs the player to remove their VR headset. During the entire experience, spectators can watch the gameplay through a TV.

The educational value is clear: children learn that what they ingest might affect their mood, stamina and health. The fun aspect is present: immersing into a cartoonish VR world and interacting with fictional characters is exciting, even for adults.

The team

Voedingscentrum was responsible for nutrition expertise and game design insights. The development team consisted of one game designer/project manager, two game developers (I inherited the project from another developer), one 2D artist, two 3D artists, one external 3D artist/animator and an external voice actor agency.

The timeline

Prototyping started as early as May of 2018 and the last version of the game was deployed in April 2019. Although this period might seem long, development didn’t take place continuously because the development team didn’t work on this project exclusively. Therefore, it’s hard to estimate how many hours were put into development.

The setup

The VR experience was meant to be installed in Dutch museums. Besides the VR headset and controllers, walls with game art, a grass carpet and a TV (for spectators) would be installed to add a fun, immersive and inviting environment. By design, the experience should work unsupervised and require low maintenance. Windows Mixed Reality (hereafter referred as WMR) was chosen as the target platform because it offers the features we sought (VR headset with controllers and inside-out tracking) at an affordable price.

48415118_10161161378390481_3020615091566411776_o

Me playing the first setup of VoedingscentrumVR at Boerhaave museum in Leiden, the Netherlands. Source: Voedingscentrum Facebook page.

The development environment

The project was developed using the Unity engine (by the end of the project, 2018.3.11f) and C# as a programming language. On initial development stages, Unity Collaborate was used for version controlling, but later we switched to a Git-based solution hosted by BitBucket.

The versions

Two versions (1.0 and 1.1) of the game were shipped. The first one was installed at Boerhaave Museum (Leiden, the Netherlands) in December of 2018 and it showed us right away that there were some challenges we had to overcome before the game was ready for the world. The exhibition was taken down and we came back to the drawing board. After a quick break, we started the development of version 1.1, on which we tackled most of the problems the previous version unearthed.

The Challenges

As expected, some challenges surfaced during the development process and after we installed the experience for the first time. In this section we discuss how we overcame them – whenever possible. Keep in mind that this article was published months after development ceased. Therefore, some of these problems might not exist anymore. If that’s the case and you are aware of a possible solution, please leave a comment on the comments section so this post can be updated. Adding new solutions to this postmorten can help many developers that might run into the same problems we did.

Controllers

Arguably the biggest challenge we faced during the game development were the WMR controllers. It’s nothing about their build quality or responsiveness (which are both great, by the way), but we ran into a few different aspects that didn’t match our project. We did not find solutions for some of these challenges, and there’s a reason behind that. At the end, the accumulation of problems regarding the controllers was so overwhelming that we decided to ditch the controllers altogether. We used a Leap Motion to track hand movement so we could use the player’s hands as interaction interfaces instead. Nevertheless, the problems we ran into before ditching the controllers  are listed below.

Although the WMR controller has many buttons (touchpad, thumbstick, menu, windows, trigger, grab), the proposed gameplay was so simple that it only required one button. This sounds like the opposite of a problem and we didn’t expect it to be one at all, but something surfaced during playtests with children. Unless they were instructed beforehand, players had a hard time figuring out how to interact with the elements in the VR world, usually because they didn’t know which button to press. Changing the button mapping didn’t seem to solve the problem because different children were attracted to different buttons. Our proposed solution was to create either a sleeve or a case around the controller to hide unused buttons and to guide the children towards the desired one. We considered modifying the controller, removing the thumbstick altogether and even placing a banner with gameplay instructions next to the headset. None of these sounded like good solutions, but we were open to trying them out in order to allow a smooth gameplay.

The game was supposed to run unsupervised in a museum. Therefore, the entire experience should always be ready for a new game session, requiring no setup from the player whatsoever. This requirement was put into check by the controllers because when left idle, they turn off automatically and – unlike the headset – there is no way (via software or hardware) to setup the idle timer duration. Therefore, if there was enough time between game sessions to bring the controllers to an idle state, the next player would have to turn them on in order to play. We were left with no other choice but to add an instruction banner explaining how to turn the controllers on, which again, was far from ideal.

WMR controllers are turned on by pressing their Windows key for a few seconds. Since it had already been stablished that the users might have to turn the controllers on before playing, that key must have been easily accessible. That introduced an unexpected problem that didn’t surface until we installed the first version of the game for public access: the same key serves as a “home” key for the entire WMR ecosystem. When pressed, the current application is suspended and the user is brought to the WMR home, an environment that serves as a start screen for MR applications. From there, the user can have access to other applications (e.g. web browser, other games, settings, YouTube…) without any constraints at all. Unfortunately, this feature can not be disabled via software and physically disabling the key wasn’t viable because it’s the same key that turns the controllers on. Hence, we were stuck in a pit: disabling or hiding the key incapacitates the controller, and leaving it accessible gives the user total freedom over the computer.  We could not solve this problem and it was one of the key factors that lead us to the Leap Motion switch.

Finally, the controllers are wireless and no cords keep someone from removing them from the experience. Attaching them to a piece of hardware would add more wires to the existing headset cords and could hurt gameplay and overall enjoyment. We didn’t expect anyone to bring the controllers home as a memento and decided to risk losing them, which didn’t happen during the short period of time the installation was displayed.

Facing all these challenges, we started to question our platform choice and to search for alternatives to WMR. But other platforms either didn’t solve our existing problems or introduced new ones. Our team had some experience with the Leap Motion and we decided it was worth a shot. We took a couple of days to implement the Leap Motion and its library into the game and by the end of it, we were quite surprised. Not only using the hands to interact with the VR elements solved all of the problems we had with the WMR controllers, but it also felt way more natural than pressing buttons. We decided to keep the WMR headset but we ditched the controllers for the Leap Motion. After some tests with children, we identified some problems regarding the Leap Motion (discussed on the next section), but they appeared to be way more manageable than the challenges we had with the WMR controllers.

Leap Motion

The Leap Motion was meant to replace the WMR controllers as interaction devices in the VR world. Although the switch eliminated the challenges we faced with WMR, other characteristic issues surfaced once we started to use the Leap Motion.

Whilst migrating to the Leap Motion solution, it was clear the one of the game’s key interactions was far from ideal: throwing balls. Previously, the user could grab balls using the WMR controller’s trigger button and throw them by releasing it, using a bezier curve as an indicator of where the ball would land. Naturally, we thought that when using your hands, the user could simply grab the ball and instinctively throw it. It sounded like the obvious design choice. Although, during internal playtests, an instinctive throw was never successful and the ball would always drop during the movement. The reason became evident: a natural ball throw usually requires a backwards arm movement, bringing the player’s hands outside of the Leap Motion tracking area. Screenshot 2019-07-07 at 17.11.39In order to execute a successful throw, the user had to keep their hands in the tracking area during the entire movement, which did not look natural whatsoever. We overcame this obstacle by changing the ball throwing design and adding a ball cannon which throws balls inserted into its feeder. After some external playtests, we added some visual clues to the ball cannon (on the right) and it was clear that the ball throwing problem was solved. Children are naturally attracted to the cannon and the visual cues make its purpose obvious.

All playtests with the Leap Motion happened during winter, which introduced an unexpected and somewhat funny new challenge. Long sleeve shirts (wore due to low temperatures) often covered part of the children’s hands, occluding them from the Leap Motion. As a consequence, tracking was regularly compromised. It’d never happen during development because the developer (a.k.a. me) comes from a warm part of Brazil and isn’t comfortable wearing long sleeve shirts. Therefore, clothing occlusion was never present during internal playtests. This is yet another – funny – example of how even the smallest cultural differences can affect game development. We didn’t focus too much on this problem because of the following reasons. First, it didn’t seem to harm gameplay substantially because the players were still able to interact with the VR elements, despite he occasional loss of tracking. Second, a small percentage of children experienced this clothing occlusion. Third, the problem was seasonal and wasn’t present for a good part of the year. Lastly, instructions to tuck away long sleeves were added to the game instructions.

In order to integrate the Leap Motion into the VR setup, the sensor had to be fixed to the headset. During development, we examined the VR headset (Lenovo Explorer), trying to find the best place to attach the Leap Motion sensor. The gameplay focus mostly on hand movement at and above eye level. Therefore, the Leap Motion sensor should be angled on a way that is optimal for the proposed gameplay. Additionally, the sensor’s location could not block the headset sensors. Once we found the perfect spot, we attached the sensor to the Lenovo headset using regular office tape. Before shipping the game, we had to come up with a more permanent, safe solution. We ended up using glue to attach the Leap Motion to the headset. The Leap Motion never fell off the VR headset during the couple of months the installation was publicly available.

During the installation of the experience, one problem became evident: the headset the client chose (Samsung Odyssey) was different from the one used during development and it had a different sensor placement. Finding an optimal spot to attach the Leap Motion became a challenge because there’s less free room on the Samsung headset. After a few tests, it became obvious that there was no spot that would not occlude (even if just a bit) the headset sensors without compromising the hand tracking. In the end, we glued the Leap Motion in such a way that it would slightly obstruct the headset sensor, but not enough to compromise the WMR tracking. As a lesson, make sure you use the target hardware during development to avoid problems on installation/deployment day.

Sound

Since the game is intended to be displayed on public spaces, the environmental noise could muffle the game voices and sounds. Thus, a setup with headphones was desirable. Luckily, there was a WMR headset with built-in headphones: the Samsung Odyssey. During playtests, a new game requirement was added: the external TV used to display the gameplay to spectators should also play the game sounds. By the time the game’s first version was installed, there was a Windows Mixed Reality setting to enable or disable audio redirection to the headset, but no audio mirroring option was available. The audio could go either to the headset or to the TV, but never to both. After some research, we found some third-party Windows applications that could simulate audio cards and that could potentially solve the problem, but it introduced some delay between the headset and the TV audio, which was undesirable. By the time we started working on the game’s second version, the WMR settings were updated and a mirroring option was added, but it presented the same delay problem as the third-party mirroring solution. Currently, the delay is still present and we have not found a solution for it yet.

Windows Mixed Reality

Even though the WMR platform sounds really promising and the quality of the compatible hardware is quite impressive for a first generation of devices, the development platform has a lot of room for improvement.

Sadly, the WMR settings panel is quite basic and lacks many features that are present on other VR platforms (idling timeout, stop rendering when idling, “locked” mode, etc). We needed some of these features for our game and it was sad to see that they were just not there. When searching for solutions, we found out that it was possible to develop SteamVR applications for WMR compatible headsets and controllers, so we decided to give it a try. Since we were using VRTK – which supports both WMR and SteamVR – the switch to SteamVR took less than a day and it was successful. We we able not only to use the WMR hardware with SteamVR, but we also had access to SteamVR’s control panel. Both versions of the game are actually SteamVR applications.

Additionally, there’s not much control over WMR’s play area. First, you can’t customize the play area walls (e.g. color, shader). Second, you can’t choose which way is forward while setting up the area. Apparently, the WMR platform totally ignores which way you start setting up the area and chooses whichever play area side is the longest one. As a consequence, which way the player faces when the game starts might differ based on the WMR room setup process you perform. Being able to select which way is forward was crucial on this project because the physical, printed walls (seen at the picture in the intro) should match the virtual environment. As a workaround, scripts to rotate and save the player’s rotation using keyboard shortcuts were added to the game. After doing the WMR setup, we do our own VR play area rotation adjustments using the keyboard shortcuts. This problem could be avoided if the user had more control over the play area settings during room setup. Additionally, we often experience wrong floor height values and have to use the WMR standard “Floor Check” app to fix it. I believe that a simple and easy floor height check should be included during room setup, like the one from the Oculus Quest.

Overall, I found the WMR platform to be too closed to developers. Sometimes it felt like I was fighting against the platform in order to accomplish my goals. Many settings that should be available (either on the WMR control panel or via an API) can’t be changed whatsoever. That being said, I did notice that the team responsible for the WMR platform is listening to developers and is slowly implementing requested features. Maybe the platform is just not mature enough and all it needs is some time to evolve.

Kiosk mode

The game had specific requirements that made it a kiosk application: the computer should be easy to turn on and off (ideally just one physical button), the application should open automatically when the system finishes booting up, nobody should assist or instruct players, the application should run as long as the computer was on and the game should reset every time someone takes the headset off. At the time, unlike with the HoloLens, WMR didn’t have a kiosk mode, so we had to implement the requirements by ourselves.

We cleaned the target computer from anything that could interrupt gameplay: notifications and Windows Update were disabled, anti-virus and performance improvement programs were uninstalled, Wi-Fi was turn off and unnecessary startup items were removed. Then, we made sure the application would never be interrupted by energy saving features: we disabled Window’s energy saving options and the headset sleep timer [1, 2]. To prevent screen burn-in on the headset display, we added a “headset fade” component that fades the display to black when nobody is wearing the headset.

The WMR Control Panel automatically shows up once a WMR application is open. This behavior may seem logical, but it might get on the way of some kiosk applications. Two applications are being loaded and are competing for windows focus (i.e. stay on top): the Control Panel and the game. Even though the VR application is set to run on full screen mode, it might lose focus for the WMR Control Panel, which is undesired, given that spectators would not be able to watch the gameplay with the Control Panel on top of the game window. Luckily, we found a workaround on Unity’s forums that worked perfectly. Once implemented, this solution enables to developer to steal windows focus anytime. The game steals focus once its setup process (waiting for the headset to return to an idle state) is over and the WMR Control Panel never occludes the game screen.

The game should reset every time the headset is taken off so the next player gets to experience a fresh environment. We used SteamVR’s library to check for user presence via the built-in headset’s proximity sensor instead of Unity’s user presence check because the latter only changed state once the headset idled. As a consequence, if a player handed the headset to another person, the headset wouldn’t idle, the user presence variable wouldn’t change and the game wouldn’t reset. Using the proximity sensor enabled a more controlled reset process which always reset the game when players were switched.

Once the above features were implemented, the game behaved like a kiosk application that could be turn on and off using a single button (the computer’s power button) and that would work uninterruptedly for hours on end.

Playtests

Before starting external playtests, we would introduce the game and instruct the children on how to play it. In retrospect, we shouldn’t done that. Since the game would be used as an unsupervised installation, nobody would explain to children how to play the game (in the end we added some small icons with basic guidelines on the spectator screen, but it still wasn’t as informative as the instructions we gave them). Therefore, we should’ve replicated the same kind of environment during external playtests. As a consequence, we didn’t catch usability and UX problems as soon as we should have. In fact, some of the problems were only evident after the first day the game was available to the public. On future projects, we will try to mimic the real world environment on external playtests in order to catch usability problems as early as possible.

Programming

Attention: this subsection is targeted at programmers.

I tried two new features on this project: async/await and C# 7. The first one was used as a replacement for coroutines because it delivers the same features but enables the usage of return values and error handling. You can read more about the motivation behind the switch and how to use async/await in Unity here. The new version of C# brings features as nested functions, pattern matching, tuples, more expression-bodied members and out variables. These features allow for more succinct but easily understandable code. You can read more about them here and here. Both async/await and C# 7 additions proved to be worth the time investment and I’m glad I took the time to learn about them.

Tooling

Looking back, I wish we’d created more tools to automate repetitive tasks we had to perform during development. An example is taking screenshots of food items within the environment for client approval. The first time I had to take screenshots, I didn’t think that we would end up having so many approval iterations, so I didn’t implement a tool to automate it and took them manually. And every time I had to take the screenshots again, I’d think “I’m sure this is the last time. It’s not worth implementing a tool just for this”. And as one can imagine, I was wrong. Many times.

Conclusion

VoedingscentrumVR was a challenging project, mostly from usability and UX perspectives. We learned valuable lessons on how to design unsupervised VR applications, how to deal with WMR limitations, how to plan better playtests and how to design better interactions using the Leap Motion. It was a project that seemed an easy task at the beginning, but that showed us we had more to learn than expected. But at the end, we accomplished our task and developed a fun, educational VR game that children enjoy playing.

This was my first project after going back to game development. It was also my first VR project since 2016 and my first WMR project ever. Besides learning more about VR development, this project taught me a lot about project and time management and client communication. Although there were times I wish this project was over already (as we do in every project), I’m now grateful I was part of it.

As usual, if you have any questions, comments or corrections (specially corrections!), please leave a comment on the comments section. I hope to see you all on the next blog post!

Advertisements

FP: Currying vs. Partial application

While learning Functional Programming, there were two concepts that for me, were really hard set apart: currying and partial application. One day, I finally decided I would dig deeper and learn what each one represents and more importantly, why I mixed them up. In this article we’ll see what currying and partial application are, distinguish them, see how they’re used in functional languages, discuss why these concepts get mixed up so easily and present why they are so important.

One tiny thing before we start. Even though the concepts we focus are widely used in functional languages, we won’t use a purely functional language to explain them – Python. You have to trust me on this.

Currying

Currying is the process of transforming a function that takes many (N) arguments into a series of (N) functions, each one taking exactly one argument. Well, maybe doesn’t tell us much, but let’s take a look at the example code below and learn from example. The Python code above shows the definition of the mult_pow function, which takes 3 numbers, raises the first one to the power of the second, and multiplies the result by the third one. Pretty straight forward. The only way to evaluate this function is by calling it and providing the 3 arguments.

def mult_pow(x,y,z):
    return x ** y * z

print(mult_pow(2,3,4))
>> 32

Now let’s try to curry this function. As described above, we will transform this function into 3 new functions, each one taking exactly one argument. We call mult_pow_c the curried version of mult_pow.

def mult_pow_c(x):
    return lambda y: lambda z: x ** y * z

Let’s analyze this pice of code for a minute. The function mult_pow_c(x) takes one argument and will always return a lambda, which is… a function! The returned function takes exactly one argument – z – and returns another lambda. This last lambda, again, takes one argument and returns the result. Note that the outer variables x and y are part of the inner function’s closure and can be accessed by it. As you noticed, we did exactly what the definition of currying said two paragraphs ago: we turned a function that takes many arguments into a series of nested functions, each one taking exactly one argument. To put it in a more formal way, currying transforms a function of N-arity into N functions of 1-arity.

Let’s now try to use these two functions and see how their application differ. The first function call uses the uncurried version and the second one the curried version.

print(mult_pow(2,3,4))
>> 32

print(mult_pow_c(2,3,4))
>> TypeError: mult_pow_c() takes exactly 1 argument (3 given)

print(mult_pow_c(2))
>> <function <lambda> at 0x10e908050>

print(mult_pow_c(2)(3))
>> <function <lambda> at 0x10e908668>

print(mult_pow_c(2)(3)(4))
>> 32

The first call (lines 1-2) demonstrates function application on the uncurried version and behaves as expected. But when we try to run the second call (lines 4-5), invoking mult_pow_c instead of mult_pow and passing three arguments to it, we get a type error telling us that we called that function with the wrong number of arguments. And indeed we did. Remember that mult_pow_c is the curried version of mult_pow, hence it only takes one argument. When we give it only one argument (lines 7-8), it returns a function, exactly what we expected from a curried version of mult_pow. In this context, if we have a function, what’s the most reasonable thing to do with it? Apply it! We do so (lines 10-11) and we still got a function in return, which makes sense given that mult_pow originally is a function that takes 3 arguments (3-arity). Only when we apply the functions three times (lines 13-14), we get the result we expected.

Well, well, well. It looks like we just created a super complicated way of defining functions. We need lambdas, unnecessary parenthesis when calling a function and there is a big knot in my head when I think of it. At first sight, it might look like a cumbersome way of doing simple things, but bare with me along the next section so we can understand why this is a powerful tool.

Partial Application

Let’s again use a Python example to illustrate the concept of partial application. The following code defines a function to multiply two numbers – mult – and a function to double (multiply by two) a number – double. Easy peasy.

def mult(x,y):
    return x*y

def double(x):
    return mult(2,x)

Notice that the double is defined in terms of mult. This is a pattern that we’ve seen before and some call it “wrapper functions”. Another good example are the functions pow (power) and square (^2). We use previously implemented functionality and we fix one of the parameters to create a new function. This is also known as partially applying a function, simple like that. Given the original function, we limit its expressiveness by tying some of its parameters. For example, take the * operator (multiplication): initially, its image (the set of possible outcomes) includes all the integers, but when we partially apply it, by tying the first argument to 2 (*2), we reduce the function’s expressiveness by limiting its image to even numbers.

So that’s partial application. On a formal way, partially applying a function to X arguments is the process of transforming a N-arity function into a (N-X)-arity one by means of tying X arguments to values.

Currying vs. Partial Application

Even though the general idea – and even the semi-formal definitions – might seem closely related, keep in mind that unlike currying, partial application does change the function expressiveness and its meaning. Another way of seeing this is to look at both as a function, where currying would be Curry(f) and partial application would be PA(f, x…xn). Currying only takes a function as input and returns another function, whilst partial application takes a function and N (N > 0) parameters.

Also, remember the definitions previously introduced: currying is the process of transforming a function of N-arity into N functions of 1-arity. Partial application is the process of taking a N-arity function, X parameters and transforming it into a (N-X)-arity function. Now, let’s take a look on how these concepts are introduced in functional languages and why we mixed them up so easily.

Currying and Partial Application in Functional Languages

Even though we can manually curry a Python function (just like we did earlier), Python doesn’t support currying by default, which means that the following code doesn’t run, where mult_pow is the function we defined on the first section.

print(mult_pow(2))
>> TypeError: mult_pow() takes exactly 3 arguments (1 given)

That might seem logical for a Python (Or C, C++, Java…) programmer, but it makes currying – something that as we’re going to see, is extremely powerful – a manual, programmer-dependent task.

The twist is that – unlike in imperative languages – in most functional languages, functions are curried by default. This might go by as a simple detail, but it has big consequences, as we’re going to see later. In Haskell or Clean, every function is curried by default. There’s no way of escaping out of it. As a consequence, every function that takes N arguments is actually a function that takes one argument and returns a function that takes N-1 arguments, just like we did on our first Python currying example (mult_pow_c). In addition, we get that for free: there is no need for any special implementation, syntax or annotation. That, aligned with higher-order functions gives us a great toolset to play with functions.

Partial application in such languages becomes so easy because of default currying. As we saw in the Python examples, partially applying a function isn’t as trivial as in a functional language.

Take the following Haskell example in consideration (if you’re not used to Haskell, don’t stop here, I won’t use it too much). We define the function multPow that takes 3 integers as arguments and returns an integer. Remember that since Haskell functions are curried by default, multPow actually is a function that takes an integer as parameter and returns another function that takes 2 integers as parameter.

To prove that, we use GHCi (GHC’s interactive environment), to partially apply multPow to the parameter 2 by doing pMultPow = multPow 2. Due to currying and partial application, pMultPow is a function that takes 2 integers as parameter and returns another one. You don’t believe me? Ask GHCi to give you some information about pMultPow by typing “:i pMultPow” and it will tell you exactly what we expected.

multPow :: Int -> Int -> Int -> Int
multPow x y z = x ^ y * z

-- In GHCi:
> pMultPow = multPow 2
> pMultPow 3 4
32
> :i pMultPow
pMultPow :: Int -> Int -> Int

One interesting fact is that you can notice that from the function’s type annotation itself. We’re used to read Haskell function type signatures in a non-curried manner. For example, we’d read multPow’s type signature as “it takes 3 integers and return an integer”, but we might as well read it as “it takes one integer and returns a function that takes 2 integers and returns an integer”. Both are equivalent in a language that is curried by default.

The Confusion

As we saw, partial application really shines in languages that have functions curried by default because there’s no extra step necessary to define curried function as in most imperative languages. That’s exactly why we easily mix up those concepts: we don’t see the currying, it’s done automatically. So every time someone tries to give an example of currying in a functional language, they fail to point exactly what it represents without using partial application, and the confusion is born. We can avoid that by explaining both concepts using a non-functional language and then bring the discussion back to the shiny, fancy functional world. It’s also why I chose to start this article with Python instead of Clean or Haskell.

Why is it so important?

The triad of higher-order functions, currying and partial application provides powerful benefits including expressiveness, higher level of abstraction and scalability, but there are two aspects that I want to focus: reusability and modularity.

With higher order functions, functions can be returned and passed as parameters, so you don’t need to overload a method, creating 10 different instances of it that are almost equal, except for one function call. Instead, make only one instance of that (previously overloaded) function that takes a function as parameter and every time you call it, pass the desired function.

With default currying, partially applying a function is trivial, which stops you from creating many wrapper functions just to fix some of the arguments. Let’s take the famous map as an example of how these concepts increase modularity and reusability. In Clean, map can be defined as in the code below. Start is Clean’s equivalent to C’s main and function type signatures use the arrow only before the output type (not in between inputs as in Haskell).

map :: (a -> b) [a] -> [b]
map _ [] = []
map f [x:xs] = [f x: map f xs]

double :: Int -> Int
double x = x * 2

Start = map double [1,2,3,4,5]
> [2,4,6,8,10]
Start = map ((*)2) [1,2,3,4,5]
> [2,4,6,8,10]

As we can see – by the (a -> b) – map is a higher-order function: it takes one function as input. It’s also polymorphic, given that it operates on lists of any type (a and b here are type variables). So, as expected, it maps a function over a list, element by element. In addition, we define the double function which… well, doubles a number. After mapping double over a list, we get a list in which all of its elements are the double of the original elements. Notice that we don’t limit the types a and b by any means. Given that, every time I need to apply a function over a list, I use map. Due to its high-order nature, I don’t need to worry about the list type as long as I provide a fit function as argument. That means that I don’t need to write 10 versions of overloaded maps, one for each kind I need, and even for different functions to map over! Imagine creating versions of map in C for ints and doubles, for doubling, squaring, dividing by 2 and many other possible operations. In a functional language, that’s not necessary.

Finally, let’s take a look on the second map example. Instead of defining a new function just to multiply an element by two, I can take advantage of Clean’s default currying and partially apply – on the spot – the * operator on the argument 2, which would give me a new function that takes one integer as input and return an integer – exactly what I need to use map, and exactly what double is.

This is a small and naive example to illustrate one of the biggest advantages of pure functional programming. Since function outputs only depend on its inputs (and nothing else!), and since we have higher-order functions with default currying and partial application, functions are little building blocks. You can just plug them, pass them around, partially apply them, run them over a list and all. That allows a level of reusability and modularity that simply isn’t possible (and won’t ever be) in imperative languages. Functions like map and foldr are reused on a daily bases by functional programmers, along with many functions they’re written themselves. Reusing functions is the bread and butter of a functional programmer. Every time you see yourself writing code that somehow resembles some code you’ve written, you think “how can I generalize this and then specialize it when I need to so I don’t need to write many slightly different versions of the same code?”

Conclusion

Even though currying and partial application are concepts that are related, they are not the same thing. Currying is a process of transforming a function into an equivalent one and partial application involves fixing a function to one of its arguments. Partial application really shines on functional languages where functions are curried by default, and that might be the reason why functional programmers get those concepts mixed up so easily. Allied with higher-order functions, these concepts increase modularity and reusability in an extraordinary way, and that’s one of the aspects where functional programming shines the most.

If you have any comments, suggestions, corrections (especially corrections), please fell free to leave a comments below.

Sources:

I can’t remember all websites that helped me with this, but here are some:

http://www.defmacro.org/2006/06/19/fp.html
https://stackoverflow.com/questions/218025/what-is-the-difference-between-currying-and-partial-application
https://www.schoolofhaskell.com/user/EFulmer/currying-and-partial-application
http://learnyouahaskell.com/higher-order-functions

Android 6 (Marshmallow) permissions and Unity

While working on Overclock, our team faced a new problem with Android 6 (Marshmallow): all permissions were requested when the app launches the first time instead of when the user downloads the app. Not only that, but the permissions are requested individually and not altogether like it used to be, which showed us some permissions we didn’t even notice were there. After some research, we found some answers and some problems and I’ll list them here to hopefully help some developers in the future.

This article will talk about Android 6 permission changes, how Unity adds some of them based on your API requests, how the engine requests them and which changes should be made to this system.

Android Marshmallow’s permission changes

Starting on Android 6, user permissions are requested during run time rather than during installation. According to Google, the change streamlines the installation and update process and allows fine tuning of the app’s functionalities: since the permissions are requested individually, the user can allow some and deny others. For example: an user can grant location privilegies but deny camera permission and the app may function seamlessly. Before the update, given that all the permissions were displayed grouped before installation, denying them would cancel the installation.

The Android Support Library provides us with methods of checking, requesting and dealing with permissions request responses. You can check more about it here.

A bit about Android permissions

In order to fully understand the problem, we must learn a bit more about Android permissions, starting with protection levels.

Protection Levels
System permission are divided into two protection levels: normal and dangerous. Normal permissions usually request data or resources out of the app’s sandbox but that shouldn’t harm the user’s privacy or other apps, e.g., time zone permission. You can take a look at the list of normal permission here. Dangerous permissions request either data or resources that may affect the user’s privacy of the execution of other apps, e.g., read the phone’s contact list and manage phone calls. Normal permissions are automatically granted without user prompt, while dangerous permissions require user authorization.

Permission Groups
Another concept that we should talk about is Permission Groups. Given that the number of permissions available on an Android environment might be quiet long, they are grouped into Permission Groups based on which data or resource it requests access for.

Let’s take the “Contacts” permission group as an example. It contains three permissions: READ_CONTACTS, WRITE_CONTACTS and GET_ACCOUNTS. As you can see, all the permissions are directly related to the user contacts. Groups are extremely handy because if your app needs those three permissions, we don’t need to request each one individually. When we request any permission in the “Contacts” group, we are actually requesting a group permission. Given that, if an app is running on Android Marshmallow requests the READ_CONTACTS permission, the user grants it and later on it requests the WRITE_CONTACTS permission, no prompt will be shown to the user on the latter, because he/she already granted permission to the “Contacts” group. Hence the WRITE_CONTACTS permission shall be automatically granted.

Unity and Android permissions

Now that we know how Android permissions work and what changed after the latest update, let’s understand how Unity handles permissions.

Permissions Added by Unity
One of the most common searches about Unity and Android permissions is “why is my app  requesting permission to make and manage phone calls if I don’t have that on AndroidManifest.xml file?” and that’s exactly how we started our search. After some reading, we found out the READ_PHONE_STATE permission is the source of the problem.

We thought “we don’t need to read the phone state, why is that permission even in the AndroidManifest?” and proceeded to delete the permission request. To our surprise, even after deleting the entry, our app still requested the permission. We thought we were crazy for a second and started to dig in all the AndroidManifest files we have in our project (plugins) looking for that entry, but we couldn’t find it.

The villain
Then, after some more research, we found the villain: SystemInfo.deviceUniqueIdentifier. This simple example shows us how Unity sometimes might be a bit too nosy and adds some permissions to our AndroidManifest on the build process. Some of Unity’s API calls require a permission to be granted to function rightfully. Unity (trying to make your life easier) identifies when your application’s source code contains that call and adds the permission request to your AndroidManifest file during the build process (hence why you can’t find it in your Manifest file). Some might say that’s too much babysitting and some might say it’s a good feature and I personally think there should be a switch on that functionality, but let’s move on.

Now let’s understand why this call needs a permission. The deviceUniqueIdentifier call does exactly what it says: it returns the device’s unique identifier (genius!). This identifier is often used as an ID for guest users so the game can save its progress on the server, even though the user didn’t create an username. This allows users to keep their progress after deleting the app from their phone. Given the premise that every phone has an unique, non-mutable identifier (guess what, it doesn’t), this works great.

The problem lays on how the engine gets that identifier. There is no guaranteed way of getting a phone’s unique ID, but there are some solutions close enough and the IMEI is the most famous and used one. The IMEI is an – usually – unique identifier used to identify most mobile and satellite phones in the world and is generally used to block stolen phones around the globe.

The root of the problems is that to access the phone’s IMEI on an Android phone, you need the READ_PHONE_STATE permission, which is part of the PHONE Permission Group. Now try to guess how a request to that permission group is described. You’re right: “Allow this app to make and manage phone calls?”. That’s right. Just to access the IMEI, you must request permission to the entire PHONE group, which includes making and managing phone calls. It sounds like an overkill, but that’s how Android sets it up.

Recap
So we found the cause of the problem: fetching the device’s unique ID needs the READ_PHONE_STATE permission which is in the PHONE permission group. Unity, trying to help us, automatically adds the permission when the deviceUniqueIdentifier call is present in code. Based on those findings, it looked like there was no scape to that problem: we either don’t use that API call to fetch the device’s unique ID or we accept the problem and move on.

We considered abandoning the guest login option, since we had another login methods (email, Facebook, Game Center) that were also affected by the permission request problem, even though they didn’t even use the device ID. After some talk, we concluded that, from a game design perspective, removing that option was more painful than having that aggressive permission request.

But… What if?

During the talk about Android permissions and mobile games, we noticed that some of the games we played already made use of the runtime permissions, but on a less intrusive way. They only requested a permission when it was really necessary and showed a popup explaining why they needed that permission. That is far from ideal, but solves 2 problems:

  • Request permissions when it’s not necessary
  • Letting the user know why we need that permission

After some digging, we found out that Unity doesn’t provide a way of doing that natively. The engine simply requests every single permission when the app starts and there is no way you can control that. The only tool available is completely turning off the requests, which we will talk about on our workaround.

Non-official workaround
Since Unity lacks of a solution for this problem, we sought an alternative solution. We can’t control when Unity requests the permission, but we can use another tools (1, 2) to do that, totally bypassing Unity. Those tools let us work with the Android 6 permission model inside Unity. One extra step to implement that is to tell Unity to stop requesting the permissions when the app launches. We can do that easily by adding the following line to the AndroidManifest.xml file:

<meta-data android:name=“unityplayer.SkipPermissionsDialog” android:value=“true” />

Conclusion: What Unity should do

The final fix for this problem would be an official solution from Unity: an API that implements the Android Marshmallow (6.0) permission model, giving us control over runtime permission requests. Our team wasn’t the first one to bump into this problem and won’t be the last, and there is a Unity Feedback topic currently open that addresses this very same problem. If you feel affected by this, please vote for it.

Source:
http://developer.android.com/training/permissions/requesting.html
http://developer.android.com/guide/topics/security/permissions.html
http://answers.unity3d.com/questions/1152724/disable-permission-dialog-in-unity-5.html

Unity Serialization Part 3: Scriptable Objects

This post is part of a series about Unity serialization. Click here for part 1: how it works and examples or click here for part 2: defining a serializable type.

On the last article, we learnt how we can define our own serializable types and discovered some problems that can emerge from it. One solution (although not ideal) to our problems is ScriptableObjects. These objects have two major uses: saving and storing data in the Editor and saving and storing data as an asset. According to the documentation, they are optimized and can store huge portions of data. There are a few singularities about Scriptable Objects and we will discuss them on the following paragraphs.

Defining and Creating

Primarily let’s learn how to define a Scriptable Object using the same example we’ve been using in the last article:

public class MyDatabase : ScriptableObject
{
  public List<City> cities;
}

As we can see, the only difference is that our class now derives from ScriptableObject. There is no need to add a serializable modifier to our script, since ScriptableObject derives from Unity.Object and that should be enough to make it serializable. The next step is creating an instance of MyDatabase, which isn’t possible without some extra code. In the past, a mix of EditorUI and AssetDatabase code was required, but recently (Unity 5.1), a simple modifier can make our lives better: [CreateAssetMenu].

[CreateAssetMenu]
public class MyDatabase : ScriptableObject
{
  public List<City> cities;
}

By marking your ScriptableObject with this modifier, a menu item will be created in the Asset/Create menu. Easy like that, this should create an Asset called “New My Database” in the Assets folder and it can be referenced by any MonoBehaviour just like any other type derived from Unity.Obejct. By doing this, we solve the first problem present on the previous blog post: databases should be assets and not MonoBehaviours. Now let’s learn how ScriptableObjects act differently from custom serializable classes in Unity.

Objects are stored as references, not copies

Consider the example from the previous post and think about this: if a MonoBehaviour has two references to the same City object, how will they get serialized? How does the change made in one affect the other? The answer to these questions is slightly counterintuitive:  they are decoupled and serialized separately, hence changing one’s value won’t change the other’s. Let’s see an example using a MonoBehaviour that executes in edit mode just to make things easier:

[ExecuteInEditMode]
public class MonoBehaviourDecoupleTest : MonoBehaviour 
{
  public City city1;
  public City city2;
     
  private void OnEnable()
  {
    if (city1 == null)
    {
      City chicago = new City();
      chicago.name = "Chicago";
      city1 = chicago;
      city2 = city1;
    }       
  }
}

Based on this example, any changes made to the object “city2” are also reflected on the object “city1”, which happens… until you hit play and pause. After doing that, modifying one object doesn’t affect the other. That happens because by hitting play, we force Unity to serialize the scene data, which will serialize each object “in line” (copying all serializable variables into a new object) and not by reference. By doing that, Unity forgets that those two objects were actually the same and treats both as separate objects.

Only classes that inherit from Unity.Object are serialized as actual references and fortunately ScriptableObject can help us with that, since it is a UnityObject. Let’s use the same example, but first let’s define a new City class, this time as a ScriptableObject:

public class ScriptableCity : ScriptableObject 
{
  public string name;
}

Now let’s use it instead of a regular City like the previous example. Note that we have a peculiar way of instantiating a ScriptableObject, and the reason behind that will become clear soon:

[ExecuteInEditMode]
public class ScriptableDecoupleTest : MonoBehaviour 
{
  public ScriptableCity city1;
  public ScriptableCity city2;

  private void OnEnable()
  {
    if (city1 == null)
    {
      city1 = ScriptableObject.CreateInstance<ScriptableCity>();

      city1.name = "Chicago";
      city2 = city1;
    }

    Debug.Log(city1.name);
    Debug.Log(city2.name);
    city1.name = "New York";
    Debug.Log(city1.name);
    Debug.Log(city2.name);
  }
}

We use Debugs to check the data since the inspector won’t show us much useful info easily. Differently from the first example, hitting play and pause won’t decouple the references and both cities will be named “New York”. Notice the difference here: the variables “city1” and “city2” were not serialized as part of the ScriptableDecoupleTest script – which happened in the first example. What Unity actually does is create a scene object (remember the odd way of instantiating it?) and keep it hidden, serialize it as part of the scene, and save its reference in “city1” and “city2”. Therefore, every time we change one variable, the other changes too, since both reference the same object, just like any reference to an UnityObject (Rigidbody, Collider, Renderer…). Thereby, ScriptableObjects solves the third problem presented on the previous blog post: reference decoupling.

We can also use ScriptableObjects as assets instead of scene objects and the effect is the same. In addition, this approach may be considered really specific, but is essential when really complicated relationships must be consistent between objects.

Polymorphism

We face a serialization problem when using polymorphism and custom serializable classes: an instance of a derived class is serialized as an instance of the base class. Let’s use the example the Unity Documentation does: Animals. See the example:

[System.Serializable]
public class Animal 
{
  public string name;

  public Animal (string name)
  {
    this.name = name;
  }
}

Now let’s define the Dog class:

public class Dog : ScriptableAnimal 
{
  public string breed;

  public Dog (string name, string breed) : base(name)
  {
    this.breed = breed;
  }
}

And a simple MonoBehaviour that runs in the editor for testing:

ExecuteInEditMode]
public class BehaviourExample : MonoBehaviour 
{
  public List<Animal> animals;

  private void OnEnable()
  {
    if (animals == null)
    {
      animals = new List<Animal>();
    }
    if (animals.Count == 0)
    {
      Animal elephant = new Animal("Elephant");
      Dog dog = new Dog("Dog", "Bulldog");
      Animal lion = new Animal("Lion");

      animals.Add(elephant);
      animals.Add(dog);
      animals.Add(lion);
    }
  }

  private void OnDisable()
  {
    Debug.Log(animals[1] is Dog);
  }
}

If you have a list of Animals and you add a Dog to it, it will be serialized as an Animal, not a dog. Add this script to an object in the scene and observe the console when you deactivate it, you will see it displays “True”. Nice! That means that Unity knows that that object is a Dog, right? Well, hit play, then stop and do that again. Now it doesn’t remember that anymore, but what happened to that Dog? When you hit play, you told Unity to serialize the scene data, and that’s exactly what it did. It happens that because of how Unity’s serialization process works, it forgets about the fact that the Dog is a Dog and treats it as an animal instead.

ScriptableObjects can help us with that, given that it inherits from Unity.Object and that type is always serialized as references – and never inline. Let’s change the definition of Animal a bit and make it inherits from ScriptableObject:

public class ScriptAnimal : ScriptableObject 
{
  public string name;
}

Now let’s use an equivalent example to the one above, but adapting it to the way ScriptableObjects are created and initialized:

[ExecuteInEditMode]
public class ScriptableExample : MonoBehaviour
{
  public List<ScriptAnimal> animals;

  private void OnEnable()
  {
    if (animals == null)
    {
      animals = new List<ScriptAnimal>();
    }

    if (animals.Count == 0)
    {
      ScriptAnimal a1 = ScriptableObject.CreateInstance<ScriptAnimal>();
      Dog dog = ScriptableObject.CreateInstance<Dog>();
      ScriptAnimal a2 = ScriptableObject.CreateInstance<ScriptAnimal>();

      a1.name = "Elephant";
      dog.name = "Dog";
      dog.breed = "Bulldog";
      a2.name = "Lion";

      animals.Add(a1);
      animals.Add(dog);
      animals.Add(a2);
    }
  }

  private void OnDisable()
  {
    Debug.Log(animals[1] is Dog);
  }
 }

This example is analogue to the previous one, so run it and observe the console. Even after hitting play, stopping the game and disabling the script, we get “True” as output, which means that the Dog is still a Dog and it was serialized as one, keeping the polymorphism intact. Therefore, ScriptableObjects solve the second problem introduced in the last blog post: polymorphism and user-defined classes. This might not be the best solution ever when dealing with serialized polymorphic code (we can’t use constructors anymore and inspecting the objects is a pain) but it may be necessary in some situations.

In addition, a fix to this polymorphism serialization problem has been extremely requested by the community in the last years and haven’t been solved it.

 Serialization depth limit (or no support for null)

Demonstrating an example of the depth serialization problem in Unity is fairly simple. Take a look:

[Serializable]
public class DepthClass  
{
  public List<DepthClass> depthObjects; 
}

This code looks harmless, but it’s not, as seen on the error thrown (on previous versions of Unity it was a warning, or simply didn’t thrown any message at all):

Serialization depth limit exceeded at ‘DepthClass’. There may be an object composition cycle in one or more of your serialized classes.

Let’s understand why. Let’s take a look at the “depthObjects” variable and try to figure out how it would be serialized in Unity if it’s an empty list. The naive thought would say that if it should be serialized as null, like any other field, but it’s not. It happens that Unity doesn’t support null serialization for custom classes (like it does for UnityObjects) and that object will be serialized as en empty instance – which is transparent to the user. On top of that, each of those empty instances has its own “depthObjects” variable, which would be serialized as another empty instance, creating a cycle that would never end, therefore crashing Unity. To avoid that, the folks at Unity Technologies set a limit (7 levels) for serialization depth, which means that after 7 levels of depth, Unity will assume that it hit a cycle and will stop the serialization at that point. Seven levels of serialization can sound like too much, but for some problems, it might be necessary.

Given that, always think twice before serializing any field that has recursive declarations. If it’s not really necessary to serialize it, don’t! But if you really want to use 7 or more levels, or you really need to serialize it, you should use ScriptableObjects. Since ScriptableObjects do support null serialization, this problem simply vanishes:

[Serializable]
public class DepthClass : ScriptableObject
{
  public List<DepthClass> depthObjects;  
}

And this simple change should get rid of the error and let you use more than 7 depth levels of references.

Data can be shared among objects

This problem (although not listed on the previous article) is exactly the same discussed in the Unity documentation about ScriptableObjects. If you store an array of integers that occupies about 4MB in a prefab and you instantiate that prefab 10 times, you would have a copy of the array in each prefab’s instance, occupying a total of 40MB. That happens because the array lives on the script itself, and copying the prefab copies the script, hence the array.

If instead of declaring the array on the prefab’s script you had a reference to a ScriptableObject which contains the array, all the instances would point to the same data. Therefore, 10 instances of that prefab would store 10 references to the same object (and not the actual data), occupying about 4MB only. Not only that substantially saves memory, but changes made to the array from any of the instances will affect the others, maintaining database consistency.

There is a catch, though. What if, instead of storing the array on a ScriptableObject, we store it on an empty prefab with a MonoBehaviour? Let me show what I mean. First let’s create a MonoBehaviour to holds the data:

public class BehaviourDatabase : MonoBehaviour 
{
  public Vector4[] vectors;
}

Let’s add this script to a scene object and save it as a prefab. Now let’s create another script that holds a reference to a BehaviourDatabase, add it to a scene object (let’s call it DatabaseModifier) and drag the prefab we just created to it’s “database” field.

public class MemoryTest : MonoBehaviour 
{
  public BehaviourDatabase database;
}

If we copy this DatabaseModifier object a dozen times and run the scene, we see that it behaviors exactly how we expected the ScriptableObject to behave: they all store a reference to the prefab on the project folder – and not the actual data. Not only that, but any change made on the array of Vector4s will also change the prefab’s data. Well, but if that also works, so why use ScriptableObjects instead of a prefab and a MonoBehaviour to store only data as an asset?* That’s what I tried to find out, but coudln’t find any solid, convincing and direct answer. Among some reasons, I found that:

  • We don’t have a GameObject and a Transform and any overhead it can create;
  • The idea behind a prefab is that it can be instantiated multiply times in runtime, which isn’t our case;

Those are not really convincing, but I personally rather have a ScriptableObject just because of the second reason. For the other uses, though, ScriptableObejcts are extremely recommended.

*Note the emphasis on _data_only_. Using a prefab and a MonoBehaviour won’t solve the other problems.

Conclusion

In this article we learnt how to solve some problems (most described on the last article) that may emerge when dealing with custom serialized types and Unity serialization. Even though some problems can be solved by using ScriptableObjects, it’s far form ideal.

 

Unity Serialization Part 2: Defining a Serializable type

This post is part of a series about Unity serialization. Click here for part 1: how it works and examples.

On the last article, we discussed about serialization concepts and how Unity implements it, learning which types can be serialized and which cannot. But what if we want to define our own type? How can I make it serializable so I can keep its data stored?

Understanding the problem

Let’s choose a (slight biased) model to implement as our example: a script to keep all the data to an investigation game which contains numerous cities, each one containing several places. Sounds pretty easy and straightforward, so let’s do it naively by creating MonoBehaviours: one for the database, one for the cities and one for the places. That should work:

public class MyDatabase : MonoBehaviour
{
  public List<City> cities;
}

public class City : MonoBehaviour
{
  public string name;
  public List<Place> places;
}

public class Place : MonoBehaviour 
{
  public string name;
}

There, done! But wait a second, something’s wrong: I can’t just create an instance of a MonoBehaviour, it should be attached to a GameObject and I don’t want to create a one to each instance of a city or place. Something is wrong conceptually. It happens that we can’t think of that data as behaviors, because they are not. They are simply objects, just like good old object-oriented programming. So let’s go ahead and take the MonoBehaviour inheritance from the City and Place classes.

public class City
{
  public string name;
  public List<Place> places;
}

public class Place 
{
  public string name;
}

Now let’s add the MyDatabase script to an object in the scene. Something is wrong again: I can’t see the list of cities in the inspector even though the field is public and should be serialized (therefore shown in the inspector).

Screen Shot 2015-09-20 at 7.47.14 PM

Defining a Serializable Type

That happens because we didn’t define our type as serializable, so Unity won’t serialize it. We never faced that problem before because we usually deal with classes that inherit from Unity.Object (Collider, RigidBody, Animation, MonoBehaviour…), which is a serializable type. There is an easy way to do it: add the [System.Serializable] modifier to the class:

[System.Serializable]
public class City 
{
  public string name;
  public List<Place> places;
}

[System.Serializable]
public class Place 
{
  public string name;
}

That gives us the expected result:

Screen Shot 2015-09-20 at 7.52.02 PM

By simply adding that modifier, we mark our class as serializable and solve our problem. The same process is also required when dealing with structs (serializable since Unity 4.5). In addition, Unity also serializes lists and arrays of serializable types by default.

Problems with that approach

Although this looks like a great solution, there are a few problems with it. The first (but not biggest) is that even though MyDatabase only stores data, it still is a MonoBehaviour and needs a GameObject to exist. Ideally, it should be an asset that only holds data, but we can’t simply take the MonoBehaviour inheritance off the class, otherwise we wouldn’t have a way to serialize it. What if there was a serializable type just like MonoBehaviour that doesn’t need a GameObject to live on? Keep that in mind. The other problems doesn’t involve data-storing objects only like the first one, but are also valid.

The second problem involves polymorphism and happens when a class inherits from a user-defined serialized class. Even though it’s intuitive that fields from both classes should be serialized regardless, that doesn’t happen. Let’s use the same example as Unity blog does: animals.

[System.Serializable]
public class Animal 
{
  public string name;
}

[System.Serializable]
public class Dog : Animal 
{
  public string breed;
}

public class PolymorphismExample : MonoBehaviour 
{
  public Animal[] animals;
}

Even though both Animal and Dog classes are serializable and Dog inherits from Animal, if we add a dog to our list of animals in PolymorphismExample, they will be serialized as Animals, losing the Dog type and consequently its fields. What if user-defined classes supported polymorphism? Again, keep that in mind.

The third problem is related to decoupled references, which is a fancy name to something really simple. Imagine you have the same Animal example as the problem above and you add 3 animals to your array, but all of them point to the same object. Due to how Unity’s serialization works, these references are decoupled and they are serialized as three different objects, hence changes made to any of those three objects won’t affect the other two. Unity simply forgets that those objects point to the same reference, which can be devastating to systems that keep complex relation between objects of that class.

The decoupling problem happens because these fields (primitives and user-defined) are serialized “in line” since they are actually part of the MonoBehaviour’s serialization data and not a data object itself. With objects that derive from Unity.Object though, the fields are serialized as actual references to the object, and not “in line” like custom classes. What if we could use a class that derives from Unity.Object, serializes the objects as references and maintains complex relations between our objects?

The last problem is related to recursive declarations, which can generate cycles. Consider this example:

[Serializable]
public class DepthClass  
{
  public List<DepthClass> depthObjects; 
}

And a MonoBehaviour that holds a reference to an instance of it:

public class DepthTest : MonoBehaviour 
{
  public DepthClass test;
} 

How many allocations will be done to serialize an uninitialized “DepthTest” script? The intuitive answer would be 1 – a null reference – but it happens that the Unity serializer doesn’t support null references of custom classes so it creates an empty object and serializes it instead (this is transparent to the user). And since this object is created and it has a reference to a list of objects of its own type, it creates a cycle in the serialization process that should go on forever. To prevent this cycle (for real, it’s not a joke) the Unity guys picked the – magical – limit of 7 depth levels and after reaching that level, the serializer will assume that a cycle was defined and will stop serializing fields of custom classes. What if we could use a type that supports null in the serialization pipeline?

Each problem described above has a potential solution and It turns out that all four can be fixed with the same resource: ScriptableObjects. It’s not an extremely elegant or ideal solution, but it’s the closest we get from one. Since it’s a fairly long subject, Scriptable Objects are described in depth on my next article. For now, let’s just acknowledge that those problems have a common way out and if you believe you may face one of those, take a look into it.

Modifiers and Serialization

Finally, let’s summarize the modifiers involved in serialization.

  • Use [System.Serializable] on a class or struct definition if you want it to be serialized;
  • Publics fields are serialized by default, but only if its type is serializable (constants, static and readonly fields are not serialized);
  • Use [SerializeField] if you wish to serialize a private field;
  • Use [NonSerialized] if you want to avoid serialization on a public field;
  • Use [HideInInspector] if you want to serialize but not expose the field in the inspector;

Conclusion

In this blog post we learnt how to define our own serializable types and acknowledged some problems that can emerge by doing it. On the next article, we will dive deep into a resource that can work out those problems: ScriptableObjects.

Reference

Unity Manual: Script Serialization
Unity Blog: Serialization in Unity
Unity Blog: Unity Serialization
Unity Tutorials: ScriptableObject