Spin-X Engine Community - Blogs

To License Or Not To License

JarkkoL — Sat, 04 Sep 2010 13:19:56 GMT

Me Too!

It's no secret that many programmers who work in the game industry have desire to develop a game engine from scratch for the next game the company will be working on. It's a very interesting challenge, educational, nice addition to your portfolio if you manage to pull it off, the result will be exactly what you want (or so you can hope) and when you write the engine yourself you know it inside out. These are the common motivations that push programmers to develop this technology by themselves. Particularly if a team has technology leads with strong desires like these it's likely they will try to justify to the management of the company why development of their own technology instead of licensing is the best choice. These justifications may include things like:

- It's a lot of money to invest something like 1M� for a royalty-free high-profile game engine PER PROJECT when you can make one by yourself.
- There will be technological and/or production advantages when using own technology.
- The company may replicate the glory of id Software, Epic, Crytek and be able to license the technology and make tons of money off it.
- Licensing existing technology may not be exactly what the project needs thus there's need to rewrite a lot of the engine code anyway to match the requirements of the project.
- When developing the technology yourself you'll be expert with it thus gain in productivity in the long run.

These are all valid points. However, because of the personal desires to develop a game engine, it's also easy to neglect the other side of the coin.

The Other Side

Hiring programmers who have sufficient knowledge to develop a high-profile game engine from scratch is very VERY hard taking long time and if you are lucky to get some on board they come with a hefty price tag. Please don't hire inexperienced developers for such a critical task, because they will bring your project (or even company) down. Even if you license the technology and your team consists mostly of inexperienced programmers there are many ways to screw up the project anyway. Also if you plan to build a new team of experienced programmers to develop the engine it's a very risky proposition because the team doesn't have past experience in working together. However, if you have already a team of mostly experienced programmers who have successfully shipped a game project together you are in much better position.

Don't get lured by superficial eye candy engine prototypes. Getting something that looks cool on the screen is much easier than building robust engine with solid production pipeline that's essential if you want to have any hope of shipping games in time and in budget. Eye candy demos that can be put together in only a few months can give totally false impression of how much work is actually required for developing a production-ready game engine.

It'll take a lot of time to develop a game engine and pipeline to the level that is feasible for production, even for very experienced programmers with past experience in the engine development. This means that your game will be delayed by at least couple of years if you can't start the engine project a side from your game projects and tie your most talented programmers to work on it. Is that the price you are willing to pay? Don't buy into promises of having production quality engine and pipeline ready in a year. You can have SOMETHING ready in a year for production team to use, but the production team will pay the bill in productivity by struggling with a clumsy, crashing, feature lacking, immature monster, and wasting the time of your production team will cost you a lot.

If we estimate that it takes 3 years for 10 programmers to make a production-ready engine and pipeline (yes, don't forget the pipeline), don't expect that you can scale your programming team to 30 programmers and make it in 1 year. It's no news that programming team productivity doesn't scale linearly with its size, particularly when building a game engine from scratch. There is a lot of infrastructure to be build for a game engine before programmers can start to work somewhat independently thus more productively, and building a solid engine infrastructure takes a lot of time. After the initial stretch of making the engine ready for production, you'll also need to keep maintaining and improving the existing engine, so the expenses of the engine development doesn't end there but you must keep your most talented programmers tied to the engine development for good to keep it competitive with licensed engines.

Translating these estimates to the cost of an engine development, and each programmer costing something like 100k�/y for the company including all expenses, that's 3M� initial investments and 2-3 years guaranteed delay in shipping the game, followed by 1M�/y for maintaining and improving the engine. Furthermore, there's no guarantees about the quality of the engine after 3 years of initial development, unlike if you license an engine which you can at least evaluate before investing money on it. Can your programming team actually pull it off and hit the expected quality bar set to 3 years into the future? Do they have the necessary experience to back up their promises?

If it takes twice the time to develop a single game along with an engine developed in-house compared to licensing the engine and being productive from day one, what's the price of this delay? There is very good chance of missing deadlines due to many unforeseen problems in the risky technology development and production team working with unproven technology, which may lead to publisher/financier pulling the plug and thus becoming very pricy for the company. Worse, you don't have anyone outside the company to blame for the delays since it's all your work! By doubling the game development time, you have also halved the potential annual revenue your company generates from selling games. Does an in-house game engine give you such a big leverage in sales that it outweighs this loss in comparison to improving & tailoring licensed technology?

Something also to consider is that licensing a game engine doesn't only enable you to use the engine technology in your project, but also comes with several man-years worth of additional development support for your project in form of engine updates over the course of the project. Depending on the engine you may also have quite good support to get your team quickly up to speed and sort various issues in game development along the way. There may also be services provided for the engine licensees to get additional game specific features developed for it, thus providing a way to mitigate risks in your project, or options to integrate various add-on components to make the engine better fit for your project. Well built engine architectures give your programmers also a lot of flexibility regarding customizing the engine technology and tools for your games and developing technology which gives your game additional technological leverage.

Wrap Up

I don't want to say that developing a custom engine is always a wrong choice, but that it can be a very pricy one driven by personal desires of the technical staff in the company, instead of something that necessarily makes much of a business sense. If you are well established developer running multiple simultaneous game projects and can afford to dedicate experienced programmers to the engine development, it can be a good choice and pay off over the long run. Because development of a game engine is very risky and long term endeavor requiring huge investment from the game company, it's better not to underestimate the effort, money and risks involved if you decide to take that route. So if your team is thinking of developing an in-house game engine, you should carefully consider both sides of the coin, because under unfavorable conditions taking engine development route can be pretty damn expensive or even fatal to the company.

Spin-X Engine LOC progress

JarkkoL — Fri, 03 Sep 2010 11:46:04 GMT

Spin-X Engine has been in active development since late 2008 and below is a graph showing the progress in lines-of-code (LOC) since the beginning of 2009 when the engine was submitted to SourceForge SVN server for the first time. Currently the engine has a bit over 140kLOC, and as a reference UE3 had ~250kLOC for Gears of War. These numbers exclude external libraries used by the engines, such as image decoding and GUI libraries, which for Spin-X Engine add up to ~760kLOC. With the current progress the engine will reach GoW UE3 kLOC within 2-3 years. Of course kLOC number is only a rough estimate of the feature set of the engine but it gives an easily digestible sense of the size of the project.

Here's also another graph showing the frequency of SVN commits of the engine for sake of it. In January 2010 the engine was moved to a private SVN server with occassional source code submits to public SourceForge server.

Complete statistics XLSX file: sxe_stats.zip

We got a new logo in town!

JarkkoL — Thu, 10 Jun 2010 18:40:08 GMT

Finally got a new logo for Profoundic Technologies, woot! 8D Hope you like it!

Spin-X Editor level building

JarkkoL — Thu, 20 May 2010 07:53:24 GMT

Thought of making a short video about creating a level segment in Spin-X Editor for my upcoming music game. The video is a bit longer than I hoped at ~18mins even after speeding it up 2x, but I couldn't work faster, sorry :) I had all the assets already setup in the editor so the work was mostly about placing objects into the level and tweaking some parameters. Anyway, if you are interested to get the general idea about level building in Spin-X Editor, this video is a good watch.

Cheers, Jarkko

New Game Screenshots!

JarkkoL — Fri, 07 May 2010 15:36:42 GMT

Here are 3 new work-in-progress screenshots of an indie game for PC/Windows I'm currently working on. There are still many things to do, but I'm hoping to release the game during this summer. This is a music/rhythm game like DDR or Guitar Hero, where by listening the beat of the music and playing by it helps you to score better in the game. You fly through levels as this little hummingbird seen in the pictures, collecting/following things distributed by the beat of the music while avoiding obstacles in the environment. The environment also lives by the music and for example in this "mushroom" theme, mushrooms and berries in the environment flash by the beats of music. The plan is to have couple of different themes shipped with the game to choose from and more available later as DLC through the game's website. The game will have ~20 different pieces of music, and more levels created for different popular/wish list music which will be also available as DLC.

I'll release a gameplay video later once I got the game more polished. If you are registered and got comments or suggestions, feel free to write it below or send me email :)

Cheers, Jarkko

SXE in Parallel Universe

JarkkoL — Thu, 29 Apr 2010 19:07:54 GMT

In order to see what kind of multi-core scalability potential Spin-X Engine has, I tested the job queue of the engine using Intel's free Parallel Universe service. Parallel Universe portal lets you to upload executables through the web interface to the service, which runs and profiles the executables in different multi-core configurations ranging from 1 to 16 cores. After half an hour the service emails notification about the job completion with a link to the statistics page. This way you can check the scalability of simple non-interactive applications beyond the usual 4-core machines we mortal people usually have access to :)

For my test case I used a modified version of the multi-threaded Mandelbrot fractal sample shown below that's available in samples/mp_mandelbrot directory of Spin-X Engine. This is a good test case in the sense that fractal calculation is embarrassingly parallel task and well demonstrates basic multi-threading characteristics of the job-queue that's the heart of all parallelization in the engine. The test doesn't test the task dependency nor the load balancing features of the job-queue though, which are essential for optimizing the total game frame computation time in practice, but it still gives some idea how well the job-queue implementation scales with increasing number of available cores & threads. It would be interesting to write more pragmatic test case though, which would also stress these features to give better idea about the real-world scalability.
Code cpp:
#include "core/mp/mp_job_queue.h"
#include "core/w32_fsys.h"
using namespace pfc;
//----------------------------------------------------------------------------
 
 
//============================================================================
// image config
//============================================================================
enum {block_width=8, block_height=8};
enum {image_width=4096, image_height=4096};
//----
enum {num_horizontal_blocks=image_width/block_width};
enum {num_vertical_blocks=image_height/block_height};
double g_delta_cr=0.0;
double g_delta_ci=0.0;
//----------------------------------------------------------------------------
 
 
//============================================================================
// Mandelbrot fractal calculation
//============================================================================
struct mandelbrot_block
{
  double start_cr, start_ci;
  uint8 *result;
};
//----
 
void calculate_mandelbrot_block(mandelbrot_block *block_)
{
  // calculate Mandelbrot fractal image block
  double ci=block_->start_ci;
  for(unsigned y=0; y<block_height; ++y)
  {
    uint8 *res=block_->result+y*image_width;
    double cr=block_->start_cr;
    for(unsigned x=0; x<block_width; ++x)
    {
      // calculate Mandelbrot fractal pixel
      double zr=cr, zi=ci;
      double zr2, zi2;
      enum {max_iterations=256};
      unsigned i=0;
      do
      {
        zr2=zr*zr;
        zi2=zi*zi;
        zi=2.0*zr*zi+ci;
        zr=zr2-zi2+cr;
      } while(zr2+zi2<4.0 && ++i<max_iterations);
 
      // move to the next pixel
      cr+=g_delta_cr;
      *res++=uint8(i);
    }
 
    // move to the next line
    ci-=g_delta_ci;
  }
}
//----------------------------------------------------------------------------
 
 
//============================================================================
// main
//============================================================================
int main(int argc_, const char *argv_[])
{
  init_working_dir();
  w32_file_system fs;
  owner_data<uint8> image_mt=(uint8*)PFC_MEM_ALLOC(image_width*image_height);
 
  // Mandelbrot fractal setup
  double mandelbrot_x=-2.0f;
  double mandelbrot_y=-1.0f;
  double mandelbrot_width=3.0f;
  double mandelbrot_height=2.0f;
  mandelbrot_y+=mandelbrot_height;
 
  // setup image blocks and add jobs for multi-threaded test
  {
    mp_job_queue jq;
    e_jobtype_id jtid=jq.create_job_type("calculate_mandelbrot", &calculate_mandelbrot_block);
    enum {num_blocks=num_horizontal_blocks*num_vertical_blocks};
    owner_data<mandelbrot_block> blocks=(mandelbrot_block*)PFC_MEM_ALLOC(num_blocks*sizeof(mandelbrot_block));
    enum {num_fractals=5};
    for(unsigned i=0; i<num_fractals; ++i)
    {
      logf("Calculating fractal %i/%i...\n", i+1, num_fractals);
      g_delta_cr=mandelbrot_width/image_width;
      g_delta_ci=mandelbrot_height/image_width;
      unsigned bi=0;
      jq.begin_add_job(jtid);
      for(unsigned by=0; by<num_vertical_blocks; ++by)
        for(unsigned bx=0; bx<num_horizontal_blocks; ++bx)
        {
          mandelbrot_block &block=blocks.data[bi++];
          block.start_cr=mandelbrot_x+double(bx)*mandelbrot_width/num_horizontal_blocks;
          block.start_ci=mandelbrot_y-double(by)*mandelbrot_height/num_vertical_blocks;
          block.result=image_mt.data+bx*block_width+by*block_height*image_width;
          jq.add_job(jtid, &block);
        }
      jq.end_add_job(jtid);
      jq.wait_all_jobs();
 
      mandelbrot_x+=mandelbrot_width*0.05f;
      mandelbrot_y-=mandelbrot_height*0.025f;
      mandelbrot_width*=0.9f;
      mandelbrot_height*=0.9f;
    }
  }
  return 0;
}
//----------------------------------------------------------------------------
The test calculates grayscale Mandelbrot fractal of 4096 x 4096 pixels, which is split to fixed size image blocks that are sent to the job-queue for calculation and which are further delegated to worker threads by the queue. The fractal is calculated 5 times during the entire test to make the test last ~1 minute in the single core mode on the test machine. After each of the 5 iterations job-queue is synced with jq.wait_all_jobs(); call, but this should result only in some minor idle time for the worker threads during the test because the size of each task is quite small in relation to the entire fractal computation time.

I tested few different sizes of image blocks ranging from 8x8 to 64x64 pixels to get an idea about the overhead the job-queue itself adds to the task scheduling. For 8x8 pixel blocks the test spawns total of 5 x 262,144 tasks each taking ~0.04ms in average to execute, while 64x64 pixel block test spawns total of 5 x 4,096 tasks each taking ~2.65ms in average based on the test results. For games which have the time budget of around 16-32ms per frame in order to run at 30-60 fps, tasks should be targeted to execute in around 1ms to reduce the amount of idle time at the sync point at end of the frame while minimizing the scheduling overhead. I believe this is a good general goal for task size even though the job-queue implementation in Spin-X Engine seems to impose very little overhead even on tiny tasks based on the results shown below. However, in the future architectures featuring tens or hundreds of cores splitting work to too small tasks potentially suffers from contention. Spin-X Engine job-queue implementation also tries to schedule long tasks and tasks with long dependency chains to be executed early in the frame to reduce the idle time at the sync point, so reducing the overall task size isn't as critical.

Interestingly, the test case with 32x32 pixel blocks seems to perform consistently ~10% better than the other tests regardless of the number of cores. This could be due to a cache effect.

New Spin-X Engine public release

JarkkoL — Thu, 22 Apr 2010 13:09:35 GMT

New Spin-X Engine has been released to SourceForge. You can download SXE by using SVN client, such as TortoiseSVN, from SourceForge SVN repository or as a single downloadable GNU tarball. Note that this distribution includes source code and doesn't include executable, so you have to compile it yourself with Visual Studio 2008 to try it out. For changes since the previous release please see the changelist.

Rigid Body Physics in SXE

JarkkoL — Mon, 19 Apr 2010 08:55:10 GMT

For physics simulation PhysX by NVIDIA has been integrated to Spin-X Engine and will be available in the next public release. The engine also features a streamlined physics asset pipeline and tools to further adjust physics assets in the Spin-X Editor.

Collision objects for static meshes are defined in your favourite 3D modelling application. Each static mesh can be composed of multiple collision object and each object is named so that it has "CVOL" string (short for "collision volume") in the object name, which the engine interprets as being used solely for collision primitive definition. In the two images below I have split the mushroom to "cap" and "stem" collision object in modo 401 and respectively named the objects to "Mushroom Cap [CVOL]" and "Mushroom Stem [CVOL]". The 3 other objects define the 3 different LODs used for rendering the mesh at different distances and doesn't contribute to collision object definition.

Once the mesh is properly defined, it can be imported to Spin-X Editor. Upon object import different tight bounding volumes and collision primitives for the mesh are automatically created, which can be viewed by selecting the viewed bounding volume from the "BVol" drop-down menu in static mesh editor as shown below. The red bounding volumes are different types of bounding volumes enclosing the entire object (including LODs), while the green volumes display the collision primitives that were created for the collision objects defined in modo.

The primitive type (e.g. sphere, capsule, etc.) selected for each collision object depends on the primitives supported by the physics engine, which by default is set to an enclosing primitive which has the smallest surface area for each collision object. This can be overridden in object properties for each collision object individually, e.g. in the image below I have used object-oriented box (i.e. preferred shape = oobox) instead of capsule for the stem collision object. Because the collision geometry doesn't have to match the visual geometry, you can optionally create desired primitive object in the modelling app and Spin-X Engine will automatically use closest matching pimitive for it.

To test rigid body collision in the engine I created a small physics scene in Spin-X Editor shown below:

And here's a short video clip showing the scene in action:

Optimized EPOS bounding spheres

JarkkoL — Sun, 28 Mar 2010 16:45:19 GMT

For calculating tight bounding spheres for objects I was checking out various algorithms and bumped into EPOS algorithm described in paper "Fast and Tight Fitting Bounding Spheres" [pdf] by Thomas Larsson. According to the results in the paper EPOS algorithm seems to be a good alternative regarding performance & accuracy of generating tight bounding spheres, so I decided to implement the algorithm for Spin-X Engine (available in the next public release in function bounding_sphere_epos()).

In the EPOS paper there's a table describing the cost (in math ops) of projecting points to different pre-defined axes, but these can be further optimized by caching some of the results and reusing those results in the tests. The first obvious optimization is that component multiplication by 2 can be cached, which for EPOS-98 saves total of 45 multiplications with the expense of caching only 3 scalar values. Also component additions such as p.x+p.y or p.x-p.y are performed 5 times each for EPOS-98, and there are total of 6 of these addition pairs saving 6*5-6=24 additions with the expense of caching 6 values.

Regarding the sphere tightness, all the different bounding sphere approximating algorithms I checked, EPOS included, seem to work by first defining the initial sphere by trying to find an axis with maximum spread of the points on it and then iterating through the points and expanding the sphere to include the old sphere and any point that's outside the sphere. I thought of changing this idea slightly and instead linearly iterating through the points and expanding the sphere with any point that's outside the sphere, find the most distant point and expanding the sphere with this point instead. It seemed like a better idea because the resulting sphere wouldn't be affected by the order of points in the array, and also because it should intuitively result in less expansions of the sphere thus a tighter sphere. Regarding performance this approach is quite efficient since only points outside the sphere in each iteration are tested again and there are usually very few points outside to be tested anyway. Also assuming this approach results in less sphere expansions, there's less ops to be executed for expanding the sphere.

I wrote a little test with EPOS-98, which generated 100,000 different random point sets with random number of points in each ranging from 3 to 10,000 and compared the radiuses of the spheres using both approaches. In the best case the radius of the sphere using the "furthest point" approach was 3.5% smaller and in worst case 1.3% larger than by using the original "linear expansion" algorithm, so it seems like a worthy optimization to get a bit smaller spheres with virtually no cost for these bounding sphere algorithms which use the sphere expansion technique.

Font Editor, Yay!

JarkkoL — Sat, 20 Mar 2010 13:08:21 GMT

There's now a font editor added to Spin-X Editor that will be available in the next public release of Spin-X Engine! It's very easy to add your custom bitmap fonts to your application, by first drawing the glyphs for a font in your favourite graphics editing software such as Photoshop, saving the glyph image for example as PNG and then importing the PNG file to the font editor. The editor extracts glyphs automatically from the image and generates tightly packed font texture and kerning information for rendering text by using the font. In the font editor you can tweak various properties for the font in the WYSIWYG editor, such as font material, kerning attributes and glyph spacing.

Below is an example of a font texture that I created in Photoshop. This texture was created simply by using an existing font in Photoshop and with the text tool typing a single line of text listing all the characters I wanted to support with enough space between character to horizontally separate them from each other. Nice thing is that you can edit the glyphs of the font freely in Photoshop before importing them to the editor, so for the below font I used black stroke layer style on the white font to draw outline for glyphs. It's possible to split the glyphs to multiple rows as well, but I kept it all in one line so that the font baseline can be automatically robustly deduced. If glyphs are split to multiple rows it may be necessary to add a baseline marker to the beginning of each row to specify the baseline glyphs are laid on so that they line up properly vertically. This source image is over 5000px wide so it's shown quite tiny here because I had to downscale it quite a bit to show it in its entirety:

When the above image is imported to the font editor, the glyphs are extracted with proper texture coordinates and the below packed glyph bitmap is generated as the result and saved out as a TGA file. You can specify spacing between glyphs in the bitmap in the font editor settings. It may be necessary to use larger than the 1px default value if mip maps are generated for the texture and font is downscaled during rendering to avoid bleeding between characters. During import stage the editor automatically deduces proper power-of-2 texture dimensions so that all glyphs fit to a single texture atlas. The generated TGA bitmap file is then imported back to editor as shown below. If the font uses alpha channel change the texture format to use a pixel format that supports alpha, such as DXT3 shown here:

Before the font can be rendered you must create a material for rendering. I used the basic transparent material for this font since it supports alpha blending and assigned the font texture shown above to the diffuse texture slot of the material and also defined peach color for the font in the material editor. The material defines how glyphs will be rendered, so it's also possible to use custom materials to render for example normal mapped fonts. No need to worry about wrong aspect ratio in the material editor because the texture is just stretched over a cube. You can always change the "uv scaling" to match the texture dimensions if you want to fix this, e.g. change y=2 to tile the texture twice in v direction, but it has no effect in the final font rendering.

Now that the material for the font is created, it can be assigned to the material slot in the font editor and "sample text" defined in the font editor will appear in the preview window. Various properties for the font can be defined here, such as kerning precision for auto-kerning, which is used to define how glyphs are layed horizontally next to each other when the text is being rendered. Because of the kerning information extracted from the source font bitmap, for example the 'W' and 'A' glyphs in the example text can overlap horizontally and form more naturally flowing string of text. "Glyph ascii chars" defines a character string that matches characters in the font source bitmap so that when writing text the engine knows which glyph to use for which character. I just copied the string from Photoshop to this property. Currently there's only support for ASCII characters, but UNICODE support is trivial to add. "Glyph ASCII remap" defines remapping of characters as a string of character pairs, e.g. if a font texture defines glyphs only for capital letters to save texture space, then adding character pairs "aAbBcC" will map low-case 'a' characters to use glyph defined for capital 'A' character, etc. Because I defined glyphs for all characters I left this property empty. If there's no glyph defined for a character, the character will be skipped.

The bitmap font and the font material can be loaded to your application with the regular load_object() calls for both and rendered with the exposed interfaces.

So, that's about it and happy fonting! :)

My first screenie

JarkkoL — Fri, 12 Feb 2010 22:20:25 GMT

Here's the first (and early) screenshot of the indie game I'm working on, rendered in Spin-X Editor. The gameplay still remains a mystery, but it should become more clear in the upcoming shots (:

[hi-res screenshot]

Going Crazy...

JarkkoL — Tue, 09 Feb 2010 11:23:27 GMT

... with Crazybump - in a positive way :) Have been doing some texture/material work and Crazybump seems to produce very nice results often even if you start normal map generation from a plain color texture. NVIDIA normal map plugin for Photoshop doesn't work with 64-bit version and is very limited in the way you can control the normal map generation anyway. I also tried xNormal that works with 64-bit Photoshop, but it has almost no control for tweaking the normal map generation. I usually keep Photoshop open while generating/processing normal maps/diffuse/occlusion textures in Crazybump and copy the images to PS via clip-board for some touch ups, composition & resizing. Crazybump seems to be well worth the money, so if you need to produce normal maps you should definately check the evaluation version if you haven't yet.

Support for modo LXO

JarkkoL — Sat, 30 Jan 2010 20:03:51 GMT

Seems that Luxology guys brought LightWave format with them when founding the new company since the LXO format is pretty much identical to LWO format :) Just changed LWO loader in Spin-X Engine to accept "LXOB" signature and rest of the loader is identical. Of course the loader skips all the fancy bits stored in the LXO file, but it's still able to load basic geometry. Clever move from Lux guys in the sense that adding basic LXO support to existing software is a breeze if it already supports LWO.

Starting an indie game project

JarkkoL — Wed, 27 Jan 2010 15:04:29 GMT

So I finally started to work on a small indie game I have had in plans for some time now, using Spin-X Engine of course :) The engine has been matured to a level where it makes sense to start with a small game project and improve the engine as needed. I hope to keep the scope of the game fairly small of around 100 working days, but let see how it turns out as I have never completed a full game all by myself, so just have to see how many schedule stretching unforeseen issues I'll face. I try to release some shots/videos of the progress in my blog as well so stay tuned :)

Test post

damsku — Mon, 18 Jan 2010 22:35:39 GMT

Hello

Nice little editing program here...