Question: best CPU for compiling? (Android source code)

[2timer]

Hello everybody,

I'm new here. I'm active on some Android forums but just recently found this site. I have a question, still very much hypothetical at this point, so forgive me if I can't explain all the specifics. In terms of a CPU which compiles source code, one I would like to use specifically for custom Android ROMs, what should I go for- more cores, higher clock speed, or the ever relative IPC metric? Would this be an area in which Bulldozers modules might actually excel, do you think? That would be awesome. I haven't done any compiling yet, so I really have nothing more than speculations ATM.

Thank you.

 

[RavenSEAL]

Ehhh, that is a great question.

My guess would be IPC, which in this case I'm gonna suggest SB/IB.

 

[soccerballtux]

Android ROM compiling can definitely max out multiple cores, had no problem maxing out my 4 when compiling CM7.2 for fun.

This does seem like something Bulldozer could be very competitive at. Hard to say.

 

[2timer]

Ehhh, that is a great question.

My guess would be IPC, which in this case I'm gonna suggest SB/IB.

Ok thank you.

 

[PhoenixEnigma]

Looking at the nearest thing in Bench, it looks like SB > PII > BD. A Phenom II X6 might not be a good way to go if your only concern in performance per dollar and that's your main workload.

Myself, I'd still be very tempted by SB/IB just for the all around performance, but it's $50-$100 more.

 

[Absolution75]

Hyper threading actually has a big difference when compiling code actually.

I'd try for a 2600K if you can get it.

 

[davel]

I would go with i7, and will need lots of ram. I have done it on my laptop that is an i5, and takes several hours to compile, I forgot exactly I usually just hit it and walk away

 

[soccerballtux]

Hyper threading actually has a big difference when compiling code actually.

I'd try for a 2600K if you can get it.

+1 lots of mispredicted branches in compiling code, lots of improvement to be had with HT....

 

[BrightCandle]

With my own tests of compiling Java and Scala I have determined that:
1) An SSD really makes a difference. All those small files combined with small library files means that the compiler does a lot of IO. HDDs often are the bottleneck but an SSD wont be, even a low end Intel 80GB G1 will keep up, you just need more IO.
2) Most compilation like with Maven and Eclipse is single threaded. The Scala compiler can actually utilise 2 cores but it doesn't really utilise more than that.
3) Compilation speeds once you have an SSD are driven massively by clock speed. You get a linear increase in performance.

Android specifically might be different, its been quite a while since I did that but I haven't seen Eclipse utilise multiple cores for its compilation so I suspect its only using 1 core just like with everything else.

 

[Gryz]

2) Most compilation like with Maven and Eclipse is single threaded. The Scala compiler can actually utilise 2 cores but it doesn't really utilise more than that.

You don't need a compiler with parallelism. You want a make-utility with parallelism. Altering make to start up multiple parallel compiles isn't exactly rocket-science. There were flavors of make that could do that already over 20 years ago. I have no clue about today's reality, but I sure hope every make-utility in existence today can do parallel compiles.

 

[tynopik]

E5-2687W amirite?

 

[cytg111]

You don't need a compiler with parallelism. You want a make-utility with parallelism. Altering make to start up multiple parallel compiles isn't exactly rocket-science. There were flavors of make that could do that already over 20 years ago. I have no clue about today's reality, but I sure hope every make-utility in existence today can do parallel compiles.

Actually I think we need pretty much everything with prallelism these days. Compilers too.

 

[Gryz]

Actually I think we need pretty much everything with parallelism these days. Compilers too.

Not really.

Source code consists of ten, hundreds, or thousands of different files with code. Each file holds from a few lines to at most several hundred lines. (Having more lines of code per file would be considered poor programming style).

I found these numbers online. They are probably not accurate at all. But they give an indication.
http://www.gubatron.com/blog/2010/0...f-code-does-it-take-to-create-the-android-os/

7200 files of C code, with 400 lines of code per file.
5600 files of C++ code, with 300 lines of code per file.

All these files will be compiled independently. Once they are compiled into object files, these object files will be linked into executables and libraries, etc. The compiling of these files can be done in parallel. Different executables and libraries can be linked in parallel too. With large numbers like several thousands of source files, a make-utility that keeps 8 compiles rolling at all times can keep an 8 core (or 4 HT core) cpu busy. To be sure you will always have enough workload, maybe configure your amount of parallelism to 12 or 16.

Starting parallel compiles is easy. Making a parallel compiler is less easy. I like biggest bang for your buck, with the least amount of increased complexity. Parallel make-utilities give that. Creating multi-threaded compilers will buy you no advantage.

Gmake seems to include parallelism.
http://www.gnu.org/software/make/manual/make.html#Parallel
Although the cave-ats seem to indicate that it's not very robust. You need to be more precise with your dependencies. I can't tell you more, as I lack any real-world experience in the last few years.

So if you are really worried about compiler-performance, I'd go with as many cores you can get, with hyper-threading too.

 

[2timer]

+1 lots of mispredicted branches in compiling code, lots of improvement to be had with HT....

With my own tests of compiling Java and Scala I have determined that:
1) An SSD really makes a difference. All those small files combined with small library files
means that the compiler does a lot of IO. HDDs often are the bottleneck but an SSD wont be, even a low end Intel 80GB G1 will keep up, you just need more IO.
2) Most compilation like with Maven and Eclipse is single threaded. The Scala compiler can actually utilise 2 cores but it doesn't really utilise more than that.
3) Compilation speeds once you have an SSD are driven massively by clock speed. You get a linear increase in performance.

Android specifically might be different, its been quite a while since I did that but I haven't seen Eclipse utilise multiple cores for its compilation so I suspect its only using 1 core just like with everything else.

Wow cool thanks. So I'm thinking, maybe a i3-2100?

You don't need a compiler with parallelism. You want a make-utility with parallelism. Altering make to start up multiple parallel compiles isn't exactly rocket-science. There were flavors of make that could do that already over 20 years ago. I have no clue about today's reality, but I sure hope every make-utility in existence today can do parallel compiles.

 

[2timer]

So you guys said that HT and branch prediction makes a difference. Also the IO SSD suggestion and parallel make-file utility suggestions, thank you.

So here's another query, I'm sorry I'm asking so many questions I guess I am a newb, thank you for bearing with me. I want to do this mainly as a fun hobby and prefer to do it cheap, if u know what I mean. My friend has a 40GB SSD he said he would sell for cheap. I also have a motherboard that supports Pentium 4 but not Sandy. So here's my thought: should I make a setup with. P4 and Hyperthreading plus ssd for approximately $50, or should I get a i3-2100, for $100 plus a new mobo? Wouldn't a single core with HT do well or no.

 

[tynopik]

So here's my thought: should I make a setup with. P4 and Hyperthreading plus ssd for approximately $50

NO

or should I get a i3-2100, for $100 plus a new mobo? Wouldn't a single core with HT do well or no.

i3-2100 is dual core

i3+SSD if you can

 

[BrightCandle]

So you guys said that HT and branch prediction makes a difference. Also the IO SSD suggestion and parallel make-file utility suggestions, thank you.

So here's another query, I'm sorry I'm asking so many questions I guess I am a newb, thank you for bearing with me. I want to do this mainly as a fun hobby and prefer to do it cheap, if u know what I mean. My friend has a 40GB SSD he said he would sell for cheap. I also have a motherboard that supports Pentium 4 but not Sandy. So here's my thought: should I make a setup with. P4 and Hyperthreading plus ssd for approximately $50, or should I get a i3-2100, for $100 plus a new mobo? Wouldn't a single core with HT do well or no.

You'll be much better off with the i3 because of the CPU performance. The SSD does help and it is sometimes the bottleneck (making jars are especially IO heavy) but you'll find often that your core will max out. The i3 has significantly better single threaded performance than the P4. Not to mention Eclipse is a bit of a CPU hog itself.

 

[LoneNinja]

So you guys said that HT and branch prediction makes a difference. Also the IO SSD suggestion and parallel make-file utility suggestions, thank you.

So here's another query, I'm sorry I'm asking so many questions I guess I am a newb, thank you for bearing with me. I want to do this mainly as a fun hobby and prefer to do it cheap, if u know what I mean. My friend has a 40GB SSD he said he would sell for cheap. I also have a motherboard that supports Pentium 4 but not Sandy. So here's my thought: should I make a setup with. P4 and Hyperthreading plus ssd for approximately $50, or should I get a i3-2100, for $100 plus a new mobo? Wouldn't a single core with HT do well or no.

P4 is antique garbage, just because it has Hyperthreading doesn't mean it'll do better than a modern processor that lacks it. The I3 will be several times faster, well worth the extra money.

 

[2timer]

Thanks a bunch guys. Now, all I need is to find a cheap used i3-2100 lol...

 

[BallaTheFeared]

I'm confused /\

 

[ThatsABigOne]

So he is banned?

 

[Hatisherrif]

Hyper threading actually has a big difference when compiling code actually.

I'd try for a 2600K if you can get it.

People were talking the same way about Pentium IV Anyways, real cores>virtual cores in things that require raw juice.

On the other hand, why was this user banned?

 

[Gryz]

Actually, if it's just a hobby, I wouldn't worry too much about performance. And I wouldn't spend a cent extra.

Typically you compile a whole tree only once. Does it matter if that takes 1 hour or 4 hours ? Just start the build before you go do something else. Or before you go to sleep. Then when you play with the code, make changes, recompile, rebuild and test, that will take very little time. In the order of a few seconds, tens of seconds maybe. There is really no need to invest a lot of money in a machine to build software. Imho at least.

I have no idea what time it requires to do a full Android build. According to that blogpost I linked earlier, Android has 12 million lines of code. Long time ago, I worked on software that had ~10 million lines of C code. When you compiled a new fresh tree, it would take a few hours to compile. (4 Hours, if I recall correctly). That was on a SparcStation with a 100 MHz cpu and 128 MB of RAM. How much faster are current day CPUs ? A factor 30 just in Herz. Plus improvements in IPC. So what took 4 hours to compile 10 million lines of C code is now a matter of minutes. When taking a 4-core CPU, such a compile should be doable under 1 minute.

This napkin-math makes me suspect that BrightCandle is 100% correct, and that compilation today totally depends on IO-speeds. So don't buy a special CPU. If you insist on spending extra money for you new hobby, it's better to buy an SSD.

 

[BrightCandle]

Java/Dalvik != C or C++ in terms of compilation and toolset whatsoever.

Ideally you want to be compiling and testing continuously as it gives you the best feedback, there is never enough performance. Eclipse like I have said does not use multiple threads for compilation and that is the android build and test environment, it doesn't use make at all. Eclipse does provide incremental builds so most of the time you don't compile everything anyway but you do it often enough that the lack of multithreading is a real problem when you do have to. Modern IDEs also take lots of RAM (several GBs at least on a moderate project) and use considerably CPU time to do things like refactor and the faster it happens the better.

Occasionally I do development on my core 2 duo laptop, and its painful compared to my current desktop. Scala code is especially bad as the compilation process is quite slow anyway, going back so many generations in CPU performance really hurts.

I've used Androids build process for more that just toy programs and I use these tools daily. I want more multi threading, but its a challenge without cutting up the project manually.