quinta-feira, 3 de novembro de 2016

So, I need a prefab system.

For some time now, I’ve been able to load resources from meta files, which allowed me to change my resources without the need of code recompiling. As long as I didn’t need any new kind of resources or attributes, at least. In fact, I do need to recompile when I want to create or change my testing entities, their components or their attributes, and this is getting really annoying.

I could use a scripting language to build my objects on the fly, but in the end, making a prefab loading system seemed a better choice. The reason is simple: I won’t have any early glue dependency with any kind of scripting language, which is something I want right now.

So… where do I start? A prefab system is a whole scary system – that’s why there is the word SYSTEM in there.

As usual, when something is so huge that we can’t easily choose a starting point, and the team (of one) size is small, or available (free) time is too short, a good place to start is deciding on behavior, desired features, and then applying some constraints and compromises. Having said this, here is the rule set I came up with:

  •         [r1]: Prefabs can be loaded from any type of yaml string;
  •         [r2]: A single yaml string must have a single prefab;
  •         [r3]: I must be able to load prefab’s yaml from files. A file must be guaranteed to have only a single prefab;
  •         [r4]: Prefabs can be a hierarchy of entities. There must be a single root entity, and the hierarchy will be built on top of Transform components (warning: transforms are optional components on the entities. A single, transformless root, is valid. Hierarchy without transforms on all entities should be impossible);
  •         [r5]: Prefabs components and entities can only make references to other components of the same prefab;
  •         [r6]: Prefabs can make references to another prefab only if referring to its root entity (components would be nice, but I’m making a compromise here);
  •         [r7]: Any real game entity can only make references to a prefab root entity.
  •         [r8]: Prefabs won’t be manipulated at runtime, and thus can only have “clone entities” instantiated;
  •         [r9]: A prefab must have a special (not yaml) representation in memory. This representation must not be bound to any loading type or persistent storage type / technique. This is the actual “prefab” in memory.
  •         [r10]: When instantiating a “clone entity”, all references must be properly set, and all loading events of the components must be properly called.


Rules r1, r2 and r3 are straightforward in making me dependent of YAML. Limiting a single prefab per yaml “complete” string, and a “single” yaml string per file, I reduce some flexibility of information loading in order to speed up feature development – by means of simplification.

Rule r4 says that I can have prefabs that are actually a hierarchy of one or more entities. However, the topmost entity (the root) must be a single one - so, no two-or-more-headed-prefabs. This allows some flexibility in the system, while the single root constraint will simplify a lot features that are about prefabs referencing, such as rules r6 and r7. By allowing other prefabs and entities to refer only to roots, a lot of the complexity about many special situations is immediately cut out from the system.
Rules r5 and r8 are sanity rules, intending to cut out even more flexibility in order to simplify the development. Prefabs changing on the fly can induce headaches from situations such as roots suddenly missing some components, or having absurd values on attributes. Also, if prefabs had dependency on specific components of other prefabs, the indirect dependency would result in special cases when instancing the actual entities.

Rule r9 is very important, as it will force the prefab system to have its own way of storing the prefab in memory. Think of this as the “core” – the way of loading the prefabs are simply outer layers of this core, and thus I can change the technology without needing to change the prefab system itself. If I succeed in being smart enough, it might be possible to make an internal representation that can itself be changed without affecting the engine.

Finally, r10 is the whole point of doing all this. Looking back, this system sure will need a loooot of planning before going to coding phase. I’ll have to postpone this a bit more, in order to have ideas on how to structure this on classes and, maybe, interfaces. Please, do note, I didn’t make a single rule about “being fast”. I’m new to planning this kind of system from scratch, so it feels like “being fast” would instantly lead to premature optimization.


Hope I don’t overengineer this…

segunda-feira, 17 de outubro de 2016

Hierarchical Transforms

After I made a simple Transform component to handle positions, scales and rotations, I stopped working on it in order to code other things. Then, came the situation where I needed hierarchical transforms, if I wanted to keep other components simple – 9 patches graphics moving around, for example. Basically, the idea is to have a “parent transform” in which “children” will inherit its position, scale and rotation properties.

So, how can that be done?

Searching around, I found out that there are two simple ways of achieving this. One is simply recalculate all children whenever a parent has been changed. It’s creepy. The other way is setting a dirty flag, and then recalculate the parents and children as their information is needed. This is the one I did!

Whenever a transform has one of its properties changed, it sets itself and all children, recursively, as dirty. When something requests information such as position, it will check if the transform itself is dirty. If it is, it requests the parent information so that its own information can be built accordingly to the current hierarchy, and then sets itself to not dirty anymore. This is a recursive behavior.

In order to avoid too many recalculation requests, each transform keeps a copy of it’s current, “resulting” transformation matrix. This increases memory usage and also the chance of cache misses but, right now, it’s very simple to understand and use. One drawback, though, is that children and parents can’t be easily reprocessed in parallel.


So far, performance is acceptable. I mean, I have nothing on my game scene….

quarta-feira, 28 de setembro de 2016

Inversion of Control? Ok! Gonna try!

These past weeks a friend of mine has been sending me lots of links to videos, articles and posts about Dependency Injection and, naturally, inversion of control. It turns out that a week ago, I finished the very, very very minimal code needed to start Suna in c++:

  • A LocalHost class
  • A Channels class
  • An unreliable communication channel class
  • A wanna-be 3way handshake for starting connections on “application level”, by using UDP.


There’s enough stuff in there to start getting lost as I forget the code, but since it IS still small enough, I’ve decided to try to use a few of the concepts from inversion of control. Took a while to refactor nearly everything:
-       
  • ISocket, IEndpoint, IHost, INetworkPacket, Ietcetc. So many interfaces…
  • Constructor arguments, constructor arguments, constructor arguments….
  • Now it’s really easier to change stuff around.

      Overall, I’m liking the results even though I’m hating all the boilerplate code. It became clear that I need some kind of injector to spawn instances of my concepts, so later I’ll write some kind of abstract factory. Suna is meant to be a really small project, so probably a “poor man dependency injection” will be enough.

segunda-feira, 19 de setembro de 2016

“Posting only relevant stuff on the blog? You mad?!”

The title is the gist of it. I’ve always thought that I should only post stuff that seems relevant – to me, at least – around here. However, reading random things about devblogs, one comment made me notice one thing: it’s not about relevant contents at all. Actually, it is more about showing that we, the developers and teams, care about our projects, and are still working on them. Of course, relevant content is still better, but it’s not a rule.


That being said, it means I can post about anything that I do over the little time I have available to advance with the project. And that’s what I’ll start doing. I hope. Anyway, the c++ game engine now has a name: Nest. The game itself will be called Valsendor, probably. The network api has received the name Suna and has a mascot which I can’t draw. Naming things is hard, very hard.

quarta-feira, 1 de junho de 2016

Games in C++ can't be that hard, right? - YAML.

These last weeks I've been making some progress with the components and rendering system of the game engine. It is all so crude, but it's working and that makes me happy. I've managed to get a moveable first-person camera with a cube in the scene, reduce the amount of code needed to add a component to an entity and even got both orthographic and perspective camera projections!

Now, indeed, reinventing the wheel is totally unproductive. With the amount of code I've written here I could have doubled the amount of features in my last game prototype made in Unity. In this c++ project, I'm still at cube point.

Anyway, a friend has suggested me to try to make my rendering system in Vulkan – an idea that I grew fond of. But first, there are other things I want to do, and one of them is to avoid “rebuilds” every stupid test change such as screen resolution or, guess what, switching between OpenGL and Vulkan. As a starting point, it seems like a good idea to reduce “rebuilds” by tackling two areas:

- Components, which can be coded by scripting languages such as FalconPL or Lua
- Assets data.

Components scripting and FalconPL / Lua is too far away from me right now; I simply don't have any clue on how to set up them with my engine architecture and bind everything, and I can't tell if FalconPL is still alive as well. That leaves me figuring out only how to load assets data. It could be done with a scripting language, too, but the truth is that any kind of file reading technique will do.

In the past, in a restaurant simulator prototype, I've used JSON to store data for things such as food recipes or employees names. In that, there are many advantages like being able to store rules anywhere that allows storage of text, and they are readable enough to be written with any text editor. It'd be nice if these features were available for the engine assets information. If I write down what data I need, for example, to set up a texture and a material, I'd get something like the following:

Texture:
    - Name : HappyFace
    - File : HappyFace.png
    - WrapMode : Clamp

Material:
    - Name : UnlitHappyFace
    - Texture : HappyFace
    - Shader : TextureUnlit

It is really easy to write and to read, and it is also YAML. Sure, loading times will be way longer than if I were reading binary data, but then I can simply write a “stupid compiler” to make the binary files from the texts – if I ever come to need this performance. To read the files, I've used a small but elegant library called Yaml-Cpp. The rendering configuration file seemed a nice “hello world” practice. Here's how it turned out:

Video:
    - ScreenWidth : 800
    - ScreenHeight : 600
    - Renderer : OpenGL

The Yaml-Cpp is really easy to use. The code is clear and straight-forward, even with some (not all) error precautions. With some practice, the code should get even simpler.


void GameSettings::Load(std::string fileName)
{
    YAML::Node mainNode;

    try
    {
        mainNode = YAML::LoadFile(fileName);
    }
    catch (std::ifstream::failure e)
    {
        throw Engine::FailedToLoadException(std::string("Failed to load game configuration: file not successfully loaded."));
    }

    if (mainNode["Video"])
    {
        YAML::Node videoNode = mainNode["Video"];
        for (const auto& node : videoNode)
        {
            if (!node.first.IsScalar() || !node.second.IsScalar()) continue;

            if (node.first.as<std::string>() == "ScreenWidth")
            {
                m_ScreenWidth = node.second.as<int>();
            }

            if (node.first.as<std::string>() == "ScreenHeight")
            {
                m_ScreenHeight = node.second.as<int>();
            }

            if (node.first.as<std::string>() == "Renderer")
            {
                std::string renderer = node.second.as<std::string>();

                if (renderer == "OpenGL")
                {
                    m_GameRenderer = GameRenderer::OpenGL;
                }
                else if (renderer == "Vulkan")
                {
                    m_GameRenderer = GameRenderer::Vulkan;
                }
            }
        }
    }
}

Now I'll try to think in a way to store the assets meta data in disk. Again, I'll probably end up with something like in Unity 3d.


The code has been highlighted by hilite.me.

domingo, 8 de maio de 2016

Games in C++ can't be that hard, right? - Code refactoring already.

It's been some time since last post. I've been tying pieces of code together as I programmed a way of rendering 3d meshes with Open GL. One awful point of the “learning-as-you-go” way of thinking is that, even if each small part is simpler to learn and understand, binding everything together can become a real problem as the communication between classes wasn't properly analyzed and decided upon.

So, you get pieces of code like this:

auto entity = entityPtr.lock();
entity->AddComponent(std::make_shared<Engine::ComponentReferenceStoringFactory<Engine::Transform>>(compFactories->GetFactory<Engine::Transform>()->Instantiate(), Engine::Global->GetComponentTypeMap()->GetTypeId<Engine::Transform>()));
entity->AddComponent(std::make_shared<Engine::ComponentReferenceStoringFactory<Engine::MeshFilter>>(compFactories->GetFactory<Engine::MeshFilter>()->Instantiate(), Engine::Global->GetComponentTypeMap()->GetTypeId<Engine::MeshFilter>()));
entity->AddComponent(std::make_shared<Engine::ComponentReferenceStoringFactory<Engine::MeshRenderer>>(compFactories->GetFactory<Engine::MeshRenderer>()->Instantiate(), Engine::Global->GetComponentTypeMap()->GetTypeId<Engine::MeshRenderer>()));
entity->AddComponent(std::make_shared<Engine::ComponentReferenceStoringFactory<Engine::SimpleControlBehavior>>(compFactories->GetFactory<Engine::SimpleControlBehavior>()->Instantiate(), Engine::Global->GetComponentTypeMap()->GetTypeId<Engine::SimpleControlBehavior>()));

A lot of time has been used to tie things together, as the snippet above can show. And another good deal of time will be needed to refactor and create utility classes and functions so that the system itself does not become painful.


Lesson learned: plan (more) before you code. C++ can become quite verbose.

terça-feira, 22 de março de 2016

Games in C++ can't be that hard, right? - Handles: hey freak me out, I'm coding!

About fifteen days ago I've posted about smart pointers and ownership. After laying out the rules for a piece of the system I wanted to code – namely, a way to iterate over instances of a type, following some restrictions and demands – I ended up concluding that maybe I should try to use handles. With the little time I have to code side projects, this ended up being a heavier task than expected. Because of c++ stuff I didn't know, didn't remember, didn't understand, or any permutation of these, I had to throw way many attempts.

Soooooo… the truth is, things got too complex and I decided to fall back into KISS (Keep it Simple, Stupid). Planning a bigger scope set of rules and demands would take too much of the little time available, so I've used a set of rules for the containers themselves, with handles.

[r1] – Containers are the unique owners of the data stored there.
[r2] – Contained data must be created within the container itself.
[r3] – Handles to contained data are created only (and only when) with the storing of data.
[r4] – Handles must be encapsulated by smart pointers. They must know how to resolve their destruction, and of the data they point to.
[r5] – Iterators that "directly" point to stored data must be available. They are unsafe and should be used with caution.
[r6] – Data can be destroyed even when their handles are still alive. This will make the handles point to a specific "No Data" element.
[r7] – Containers must assume "non pointers" template data types.
[r8] – Containers must be inheritable from an abstract container.

I know some rules can be rather controversial, but I'll leave it as is for now. Actually, they'll force some restrictions on the stored data as well. The biggest problem probably comes from rule 8; there are reasons that I want a uniform representation of any possible container for the same data type, such as a list of containers to be used by factories.

A good name for the container might be StoringFactory. After researching for a while, it seems to be rather unusual to mix up a container together with a factory, as the separation in "container only" and "factory only" classes is desirable. However, for my needs, it seems to make more sense to simply merge them together, with a name to clearly show this mix of responsibilities. Am I deciding wrong? Don't know, but I'm really afraid of over-engineering this, right now...


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
template <class DataType>
class StoringFactory
{
public:
    virtual ~StoringFactory() {}

    // Deletes the data pointed by a handle. The handle won't be discarded,
    // so any holding handle to the same data will still be "valid", and
    // return noDataPointer.
    virtual void DeleteData(StoringFactoryHandle<DataType>& handle) = 0;

    // Deletes the data pointed by a handle, and the handle itself. Using the
    // same holding handle after deleting it will result in undefined behavior.
    virtual void DeleteHandle(StoringFactoryHandle<DataType>& handle) = 0;
    virtual DataType* Get(StoringFactoryHandle<DataType>& handle) const = 0;

    virtual DataType* SetNoDataPointer(std::unique_ptr<DataType> noDataPointer) = 0;

    // Returns the number of registered elements in this container. The name is not
    // size as it could be misleading.
    virtual int ElementsCount() const = 0;

    // Iterators for ranged-for loops
    virtual StoringFactoryIterator<DataType> begin() const = 0;
    virtual StoringFactoryIterator<DataType> end() const = 0;
};


The abstract StoringFactory class above is rather simple. Someone that is somehow reading this post will probably notice that there are no methods to actually insert or create data into this container. The reason for that is that I want to try using emplacement methods with variadic templates. There's a catch, though: they can't be virtual, forcing them to be in the child classes. How, then, can I insert data into the container when I have only a pointer to the base class? It turns out that it is indeed possible, by means of abstract factories or dependency injections, and some magic.

Next comes the dubious methods of DeleteData and DeleteHandle. DeleteData is rather dangerous and nonsensical, but I want the ability to remove data from memory without first needing to properly clean up anything related to its handle (such as every object that has a shared ownership of the handle). This can be useful in some very rare cases, as unloading low priority resources when in need of memory. The DeleteHandle probably shouldn't be public. Handles will call this method to destroy themselves, but I don't see any reason to not make this method protected, and the handles a friend class of the StoringContainer.

And what does the handle looks like?


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
using handle_type = unsigned int;

template <class DataType>
class StoringFactoryHandle
{
public:
    StoringFactoryHandle(handle_type handle, StoringFactory<DataType>* factory);

    inline DataType* Get() const;
    inline handle_type GetHandleValue() const; // TODO : restrict creation access



protected:
    StoringFactory<DataType>* m_Factory;
    handle_type m_Handle;
};


They're just a small container with enough information to refer back to the original data. I've restricted the handles to store handle_type (that is, unsigned int) as their "handling key". I could have used pointers, but ended up deciding on "indexes" or "keys" instead. This is a decision that'll probably be very hard to change later on... so let me pretend to be proud and sure of what I'm doing.

Finally, all that remain are the iterators types for the StoringFactory class. It is possible to make an abstract iterator type and then inherit as needed, for each storage desired, such as lists, maps, etc. This should work, by relying on Covariance, but soon many flaws of this idea would sprout everywhere. One of the problems, for example, is the need of casting between parent and child types of the iterator. Even if that is solved, templates and slicing would come right after with many other problems. The path I chose was to use the pimpl idiom, though I feel like I'm missing a bit of its point.


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
template <class DataType>
class StoringFactoryIterator
{
public:
    StoringFactoryIterator(StoringFactoryIterator<DataType>& other);
    StoringFactoryIterator(std::unique_ptr<StoringFactoryIteratorImpl<DataType>> iteratorImplementation);
    ~StoringFactoryIterator() = default;

    DataType& operator *() const;
    const StoringFactoryIterator& operator ++();
    bool operator !=(const StoringFactoryIterator& other) const;



protected:
    std::unique_ptr<StoringFactoryIteratorImpl<DataType>> m_StoringFactoryIteratorImpl;
};



template <class DataType>
class StoringFactoryIteratorImpl
{
public:
    StoringFactoryIteratorImpl() {}
    virtual ~StoringFactoryIteratorImpl() {}

    virtual DataType& Dereference() const = 0;
    virtual const StoringFactoryIteratorImpl& PreIncrement() = 0;
    virtual bool IsDifferent(const StoringFactoryIteratorImpl& other) const = 0;

    virtual StoringFactoryIteratorImpl* clone() = 0;
};


The idea was to use a single iterator "interface" class to redirect operations to a real implementation, that is storage specific. Some problems with slicing still remains, such as making clones of the iterator implementation when the "interface" is copied around. Since the implementation is handled only by the "interface" class, methods such as clone are used. Casting is still a problem (i.e when the user try to copy or compare the iterator of one type of storage with one of another type of storage). I didn't decide on any special semantics for that, but later there'll be probably some assertions for debug builds.

Some methods are still missing, such as container sizes, but this is the basic for my storing with handles system for game components. I'm also implementing an ArrayStoringFactory class, that inherits from StoringFactory, but I don't think it's code has anything interesting right now.

Let's see what happens in the next 15 days. After finishing up theses classes and adding any new needed method to the base classes, I'll make some unit tests and then the Transform component. Maybe I'll have some thoughts to write in this blog again.

All code in this post has been pretty-printed by hilite.me