Chunky by FelipeFS
Chunky by FelipeFS
GPWiki.org
It is currently Wed Oct 01, 2014 3:59 am

All times are UTC




Post new topic Reply to topic  [ 32 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Data Oriented Design
PostPosted: Tue Aug 20, 2013 6:26 am 
BANNED

Joined: Sun Jun 24, 2012 12:49 am
Posts: 504
Back in the days when I was a chap who used Game Maker, the most advanced scripting capabilities -- though very cumbersome and primitive -- were data oriented features. These were tacky features which didn't correspond much to the language's syntax, but when I began learning languages such as C++, I believed they were only cheap, minimal additions to a very poorly designed scripting language (GML). I learned how C++ syntactically supported data structures (as in struct type-definitions; classes etc.)

With reminiscence of my GML days, this data-oriented stuff was all the coolest.

I found these matters to constitute the most fascinating aspect of programming. This stuff was my passion. Understanding the math behind something like triangle-mesh collisions sure was a challenge, and not a very satisfying one at that. In contrast, working with data e.g. soups of triangles, vertices, keyframes and their associated transformations (yea I wrote a little mesh skinning thing) was quite a lot of fun and very satisfying. The math was more straightforward and satisfying too. :)

Of course these practices weren't really in the data-oriented design club at all, but my mindset was quite nearby. (The attribution of "Data-oriented design" is sort-of recent in a formal form)

I stopped using Game Maker entirely since early GM7's release; I've predominately been programming with C++ ever since. When I first started C++, the only challenges I faced against were working properly with some of the new raw API baggage (Win32, Direct3D, OpenGL etc. ), but despite these expected issues, everything went very well for me. Things were good until I became unsettled with a lack of API-independent utility, specifically for math and text rendering tasks that were free of undesirable third-party attachments or unsuitable licenses (GPWiki helped with that issue :)). As I became more familiar with common C++ programming styles and C++'s new concepts -- including pointers (addresses too, 'n stuff), templates, polymorphism, manually handling dynamic allocation without STL containers and so on -- my happiness with programming rapidly declined until programming became slow, complicated, disappointing and virtually dead for me. I was frustrated with fantastic and apparently reasonable software-engineering ideals which, ultimately, I poorly understood.

Depressing years of subtle, dull, unclear and unpleasant lessons passed
while the demise of my life's passion damaged my emotional condition...

I'm coming towards the end of my 360 degree spiral and hurling out of orbital lock for an interstellar trajectory (I hope :confused)

Lately, I've been reading about something called "data-oriented design." Awe.. I might just love programming again! Here's some material for you sexy peeps: :O

http://gamesfromwithin.com/data-oriented-design
http://molecularmusings.wordpress.com/ (to find articles: CTRL-F 'data-oriented')
http://justinliew.com/blog/?p=2890
http://gamesfromwithin.com/managing-data-relationships

A book:
http://www.dataorienteddesign.com/dodmain/dodmain.html

Relevant Excerpt
Quote:
Data-oriented design has been around for decades in one form or another, but was only officially given a name by Noel Llopis in his September 2009 article of the same name. The idea that it is a programming paradigm is seen as contentious as many believe that it can be used side by side with another paradigm such as object-oriented programming, procedural programming, or functional programming. In one respect they are right, data-oriented design can function alongside the other paradigms, but so can they. A Lisp programmer knows that functional programming can coexist with object-oriented programming and a C programmer is well aware that object-oriented programming can coexist with procedural programming. We shall ignore these comments and claim that data-oriented design is another important tool; a tool just as capable of coexistence as the rest.


Top
 Profile  
 
 Post subject: Re: Data Oriented Design
PostPosted: Wed Aug 21, 2013 2:25 am 
BANNED

Joined: Sun Jun 24, 2012 12:49 am
Posts: 504
A Much Better Compilation Than Mine


Top
 Profile  
 
 Post subject: Re: Data Oriented Design
PostPosted: Wed Aug 21, 2013 6:52 pm 
Grand Optimizer

Joined: Sun Oct 16, 2011 3:09 pm
Posts: 365
Location: Here (where else?)
I slowly start to understand what they mean with data oriented design, you have to be careful where you place your data, and how you access it.

_________________
My project: Messing about in FreeRCT, dev blog, and IRC #freerct at oftc.net


Top
 Profile  
 
 Post subject: Re: Data Oriented Design
PostPosted: Mon Aug 26, 2013 10:17 pm 
BANNED

Joined: Sun Jun 24, 2012 12:49 am
Posts: 504
My thoughts right now are someplace weird.

For some reason I have a compulsion to say

DUCK YOU

I seriously don't know... ok bye.


Last edited by Pieman on Fri Aug 30, 2013 4:29 am, edited 4 times in total.

Top
 Profile  
 
 Post subject: Re: Data Oriented Design
PostPosted: Tue Aug 27, 2013 1:52 am 
BANNED

Joined: Sun Jun 24, 2012 12:49 am
Posts: 504
Why has my life come to this? xD


Top
 Profile  
 
 Post subject: Re: Data Oriented Design
PostPosted: Tue Aug 27, 2013 6:08 pm 
Cubic Contributor

Joined: Sun May 27, 2012 6:01 pm
Posts: 67
From the few articles I read, this is about optimization.

Dealing with caches, co-procossors and memory access times/layout is all architecturally specific.

OOP groups conceptually relating data, and the procedures required to manipulate that data. During development, this is incredibly useful. IMO Id stick with oop and consider this later in development.


Top
 Profile  
 
 Post subject: Re: Data Oriented Design
PostPosted: Wed Aug 28, 2013 6:19 pm 
Bibliotherapist
User avatar

Joined: Wed Nov 03, 2004 1:28 pm
Posts: 7114
Location: Wilts, Englandshire
Premature optimization is a curse on development. I've often been guilty of agonizing over bits and nybbles far to early in the dev cycle, trying to align datatypes on boundaries before the logic is in place.

I appreciate what the articles say, but there has to be balance between optimal organization of data and sensible design that enables rapid development.

_________________
10 PRINT "Bad Monkey ";
20 GOTO 10


Top
 Profile  
 
 Post subject: Re: Data Oriented Design
PostPosted: Wed Aug 28, 2013 7:07 pm 
Harmlessness does no harm
User avatar

Joined: Tue Sep 14, 2004 8:37 pm
Posts: 3952
Location: Ferriday, LA, US
What Codehead said. The best plan is one that can be easily modified to adapt to unforeseen circumstances. A plan that cannot be modified WILL be thrown away -- either before the plan is put into place, or shortly thereafter, the moment something goes wrong (which it generally will).

It's also best to work from the top, and go down. Be vague at first, then when details begin to arise, then you can worry about putting them together in a way that works not only in theory, but in practice, as well. Without a clear idea of the big picture, the little details are just an endless line of traps.

_________________
Wear your sorrows with a smile!


Top
 Profile  
 
 Post subject: Re: Data Oriented Design
PostPosted: Fri Aug 30, 2013 4:24 am 
BANNED

Joined: Sun Jun 24, 2012 12:49 am
Posts: 504
I hate all of you.


So much.


Last edited by Pieman on Fri Aug 30, 2013 7:41 pm, edited 2 times in total.

Top
 Profile  
 
 Post subject: Re: Data Oriented Design
PostPosted: Fri Aug 30, 2013 6:58 am 
Bibliotherapist
User avatar

Joined: Wed Nov 03, 2004 1:28 pm
Posts: 7114
Location: Wilts, Englandshire
Pieman wrote:
I hate all of you.

I think not: http://www.youtube.com/watch?v=FgKXBJ2LZKo&t=55s

_________________
10 PRINT "Bad Monkey ";
20 GOTO 10


Top
 Profile  
 
 Post subject: Re: Data Oriented Design
PostPosted: Fri Aug 30, 2013 2:26 pm 
Digerati

Joined: Thu Sep 09, 2004 1:17 pm
Posts: 1814
Location: burrowed
Codehead wrote:
Premature optimization is a curse on development.


I dont agree. I see a lot of people pointing that out, but i think that statement's a false premise.
Optimization is important. Knowing where you can possibly optimize also is. If you're trying a new mechanic that just barely works on its own, and fails to work in a game environment due to performance issues or other bottlenecks you'll be stranded if you don't know how do fix it up. This can be applied to several areas: rendering, physics, stuff inbetween. Knowing what functionality has what kind of impact will not only make you a better developer, it will also open your eyes to some solutions or paths you wouldn't have found otherwise.
Also, being familiar with how much a certain system or mechanic might impact on the performance will help you come up with ways to write it more efficiently to begin with.

Honestly, i just hate people saying that :P It makes it sound like people are too lazy/uncaring to learn the platform they're working with.

_________________
Long pork is people!

wzl's burrow


Top
 Profile  
 
 Post subject: Re: Data Oriented Design
PostPosted: Fri Aug 30, 2013 4:33 pm 
Corpse Bride
User avatar

Joined: Tue Jul 01, 2008 11:44 pm
Posts: 2230
Location: England
weezl wrote:
Codehead wrote:
Premature optimization is a curse on development.


I dont agree. I see a lot of people pointing that out, but i think that statement's a false premise.



Codehead used the word premature, but I feel this is misdirection, as it is not the appropriate way of describing what is written above.

By definition of the word "premature" this is optimisation that is done too early, or done over zealously. Which is bad by definition. What we actually mean here is early optimisation.

Even if we replace premature with early, then "early optimization is a curse on development" is still a a sweeping generalisation that's out of touch with reality, rather like black and white thinking.

Early optimization is synonymous with forward thinking, and planning ahead. And if that is a curse on development, then I have no idea how you manage to get any software (of any reasonably complexity) completed at all.

Without optimisation your program may not run at a reasonable speed. In many cases, optimised code isn't something you just ad-lib near the end of a project when you find your current code doesn't work very well. Sometimes you need to plan this from the beginning and design data structures accordingly, eg use linear arrays vs binary trees vs hash tables? Store database records sequentially so you can retrieve whole records quickly? Or store component fields sequentially so you can search through them quickly?

So I fully agree with wzl's sentiment insofar as the gesture is a false premise.

_________________
I ain't pushing no moon buttons.


Top
 Profile  
 
 Post subject: Re: Data Oriented Design
PostPosted: Fri Aug 30, 2013 7:46 pm 
BANNED

Joined: Sun Jun 24, 2012 12:49 am
Posts: 504
For the record, cache optimization is a subject of its own.

Data oriented design involves cache optimization, but I've already known cache optimization as a subject of its own for a long time and DOD is really a different issue. To me, one of the most important aspects about data oriented design is the myth of class-oriented extensibility e.g. write an abstract "Window" class, provide virtual methods such as SetTitle(), GetTitle(). Anyone with experience should know how it is a fault to expect the only thing to worry about after conceptually designing an abstract class (in a "straightforward" way, at least) is things like writing platform-specific implementations and so on.

Ever since I heard
"I slowly start to understand what they mean with data oriented design, you have to be careful where you place your data, and how you access it."

I knew this was heading a bad direction. -_-

Then I heard these two statements:
"Premature optimization is a curse on development. I've often been guilty of agonizing over bits and nybbles far to early in the dev cycle, trying to align datatypes on boundaries before the logic is in place."
"The best plan is one that can be easily modified to adapt to unforeseen circumstances. A plan that cannot be modified WILL be thrown away -- either before the plan is put into place, or shortly thereafter, the moment something goes wrong (which it generally will)."

That just made me go nuts. Class-oriented extensibility sounds like a harmless idea. The truth is that it does more harm than good. Unless you're DOING IT WRONG (and most people are), it is a real pain in the grass to do things right. You in fact need to plan more than you do if you just focus on using procedures to work through data. In other words, with data-oriented programming, you're not ideologizing code with procedurally inaffective, merely code-oriented functions like GetTitle(). For data-oriented programmers, writing explicit procedures to directly manipulate data is more important than providing a convoluted plethora of methods which virtually construct procedures. Do you really want procedures to become more implicit? Writing a for() loop from the top-down is very easy after just a little experience. Trying to break down functionality from a flat loop into a deep stack of operation (lots of method calls) is a stupid idea.

For example, let's consider to conceptually have two different 'Window' types. One type of window can be resized and it requires the UI nested within (if any) to adapt to its new form. Another window cannot be resized.

The data in common can be put in a common place and targeted by a single routine. The data which is different can be seperated into multiple pools. If you have a list of indices which indicate which windows need to handle adjustments to their inner-contents, then you can pass through that list and perform the updates. That's straight forward. You don't need some bs like

Code:
struct Window
{
bool resizable;
};

if (window->resizable)
{
hot dawgs
}
else
{
kewl sheep
}


but rather

Code:
for (uint resizableI = 0; resizableI != resizableQ; ++resizableI)
{
JUST HANDLE THE RESIZING OF ' resizable[resizableI]; ' AND MOVE ON WITH LIFE
}


This is also much more predictable and cache friendly. Despite how you'll have a few more loops, consider how each loop will have reduced, purposeful, data-oriented code, will be simple and straightforward, will perform predictably and only touch the data it needs (i.e. cache optimization stuff). Windows that don't need to bother with handling resizing actions don't need to suffer for trying to share all their data in the same pool of windows which do (and polymorphism is just a terrible way of emulating this). Also, most importantly, you don't need to worry at all about how the data will mesh together. This makes extensibility trivial. There's really no need for abstraction or generic programming. No need for polymorphism, no need for little booleans or flags etc.

These are the best articles in my opinion:

http://molecularmusings.wordpress.com/2 ... ical-data/
http://molecularmusings.wordpress.com/2 ... sh-data-3/

Most articles on that blog are pretty great. Here I've only listed two parts -- the best and most important, though -- of the series this author is writing on DOD. The other parts aren't essential to DOD, in my opinion. Some of the design decisions he makes in further articles, to be modest, are not the best solutions you can wager with DOD.

My two favorite blogs are molecularmusings.wordpress.com and fgiesen.wordpress.com (ryg is awesome!)


Top
 Profile  
 
 Post subject: Re: Data Oriented Design
PostPosted: Fri Aug 30, 2013 9:33 pm 
Bibliotherapist
User avatar

Joined: Wed Nov 03, 2004 1:28 pm
Posts: 7114
Location: Wilts, Englandshire
Like I said, balance is required. I'm not suggesting throwing data together with no thought and hoping it will work, that's like saying you can paint a picture by throwing paint a wall.
Some decisions have consequences and should be handled carefully. Other times, you need to get the logic down to test and see what will actually give best performance.

I've been working on an Agile project at work recently and I've been surprised how having fixed targets and a need to deliver working software rather than highly optimized code make you get things done in very short order. Sure, some modules could run better, but they can be re-visited in future sprints and optimized or refactored then. Also, I find that having the big picture and being able to see everything actually working helps to refine and optimise. I suppose its the difference between the bottom-up and the top-down view.

_________________
10 PRINT "Bad Monkey ";
20 GOTO 10


Top
 Profile  
 
 Post subject: Re: Data Oriented Design
PostPosted: Sat Aug 31, 2013 12:01 am 
BANNED

Joined: Sun Jun 24, 2012 12:49 am
Posts: 504
I want to murder you with a spoon.


Top
 Profile  
 
 Post subject: Re: Data Oriented Design
PostPosted: Sat Aug 31, 2013 5:56 am 
Grand Optimizer

Joined: Sun Oct 16, 2011 3:09 pm
Posts: 365
Location: Here (where else?)
Pieman wrote:
I want to murder you with a spoon.
I fail to understand what you want from us.
Our blessing? You got it.

I have been around long enough to notice a pattern in these things. We came from C or Pascal. Pure procedure-oriented languages.
I would say really DOD friendly.

Then came object-oriented design, it was THE solution, not more big blobs of data with separate functions, no data got smart and you can integrate data and function. Then came patterns, so good, you could write them once and use them anywhere.
Them came aspect-oriented design. Separate classes was not enough you need to be able to inter-mingle their use by code weaving.
Then we had agile programming.
Now comes DOD, totally new, and so much better!!

So I am somewhat skeptical for two reasons.

1) I don't understand it, how do you write large scale complex programs by throwing all data together? I read some stuff, but it goes right down into example level.

2) How is it different from procedural programming? I mean, there is a reason we switched to objects. Are those reasons now completely void? Having smart data is not smart anymore?

3) Everybody speaks about performance. Is it important? Yes, somewhat. I mostly care about scalability. Should it be leading in methodology? I don't think so. Methodology should not matter. I can write good and bad code in every methodology.

Programming is about creating structure in chaos. It is extremely difficult. I am happy when I find a solution, any solution. DOD makes it sound asif data storage should be an integral part of the solution. I would say, only for those parts where it matters, which is between 1 and 10 percent of the code. Also, from experience I know you cannot predict where that code is until you measure it. Heck even CPU manufacturers cannot tell you how long a single instruction takes (they say "at least X, max Y, and usually Z").

So how do you design for optimal performance beforehand?

Fon't get me wrong, I fully believe DOD has good sides. Depending on the area you work in, it may be significant. Before I throw away everything I know though, I'd like to understand how it is better.
More likely, it is not universally the best solution that makes everything else obsolete, but it is an extension of my toolbox to use when the situation is right.

_________________
My project: Messing about in FreeRCT, dev blog, and IRC #freerct at oftc.net


Top
 Profile  
 
 Post subject: Re: Data Oriented Design
PostPosted: Sat Aug 31, 2013 5:19 pm 
Cubic Contributor

Joined: Sun May 27, 2012 6:01 pm
Posts: 67
Just to point out; Most comments/rule are only a generalisation, They consider all programmers. And might not apply to individuals of varying talents.

weezl, Jasmine:

The main argument against optimisation done at the wrong time is; because optimisation/responsiveness is a quality of the system, It is tied to the implementation and usage of your system. These are moving targets, and are extremely hard to predict accurately. Because optimisation has a cost, it tends to waste a lot of time and effort when it could become redundant later in the projects life.

Some programmers can fully map their projects in their minds (planning ahead); Which is perfectly fine, but my two personal problems with this is that 1) its hard to predict the effects of an optimisation (no measurements before and after) 2) I want my head to be filled with more valuable information and thoughts relating to the content of the game.

Pieman:

Ive some problems understanding your ideas on programming and/or OOP; I also dont think 'Window' is a very good example to start with. Could you think of a better one?


Top
 Profile  
 
 Post subject: Re: Data Oriented Design
PostPosted: Sun Sep 01, 2013 7:26 am 
Bibliotherapist
User avatar

Joined: Wed Nov 03, 2004 1:28 pm
Posts: 7114
Location: Wilts, Englandshire
Good points, there is no one size fits all approach. It depends on the developer and nature of the project. My next project is on an embedded system and will need to be highly optimized due to limited resources.

_________________
10 PRINT "Bad Monkey ";
20 GOTO 10


Top
 Profile  
 
 Post subject: Re: Data Oriented Design
PostPosted: Sun Sep 01, 2013 11:05 pm 
BANNED

Joined: Sun Jun 24, 2012 12:49 am
Posts: 504
Please stop discussing optimization.


Top
 Profile  
 
 Post subject: Re: Data Oriented Design
PostPosted: Sun Sep 01, 2013 11:06 pm 
BANNED

Joined: Sun Jun 24, 2012 12:49 am
Posts: 504
Codehead, may you move these posts (all of them) to a split topic on cache optimization? My remarks on optimization were auxilary and were demonstrating how it is indeed related to cache optimization (sort of a bonus), but the concept is not the crux of DOD.


Last edited by Pieman on Mon Sep 02, 2013 3:53 am, edited 1 time in total.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 32 posts ]  Go to page 1, 2  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  

Powered by phpBB® Forum Software © phpBB Group