When anandtech reviews cpus, they seem to use a standardized set of test, offering the benefit of simple aggregation of comparable results. I'd suggest that while important, this still leaves relevant knowledge out.
So what if the Intel Haswell performs e.g. 8% better at the same clock than Ivy Bridge or 14% better than Sandy Bridge when performing a set of image processing tasks in Lightroom 3.0. Lightroom 5.0 (and 5.2RC) is out. We don't know when/if Adobe engineers switched intel compiler (-switches), or how much/early they are doing low-level intrinsics/assembler for a new architecture. We don't even know to what degree Adobe relies on external libraries to do the grunt work (Intels IPP?)
In order to have some insight into this, I'd suggest to profile Lightroom 3.0, 4.0 and 5.0 (possibly minor revisions as well) on the last 3-4 intel architectures. The task might be something like "import 100 raw files, display on color-profiled screen and apply default settings, export as jpeg". Now, the algorithms have changed over the years, so comparing absolute numbers across Lightroom versions might not make sense. But the relative scaling of different Intel architectures should tell us a lot about how aggressive Adobe are at explicitly compiling/optimizing for new architectures (how much time does it take from Intel release something new before Adobe can take advantage of it, and to what degree do they benefit from it).
As is, I know that Haswell introduce major improvements in integer SIMD (just the kind of thing that image processing might benefit from, AVX2 should double the theoretical throughput of integer SSE?), and smaller but still significant improvements to floating-point SIMD (FMA should double theoretical throughtput over AVX1 for common linear filtering tasks? for all I know, Lightroom might be heavily floating-point dependent). When the measured performance of old-ish versions of Lightroom does not increase by 2x, this could be due to any mix of the following:
1. The algorithm is not perfectly SIMD limited (inherently single-threaded parts, heavy memory/disk access). Increasing vectorized performance is unlikely to ever improve this algorithm much.
2. Adobe were focusing on other things than low-level optimization for cutting-edge hardware (or the hardware were not available to them at the time of compilation). They may or may not catch up with Haswell in newer versions.
3. The Intel improvements are theoretical, other parts of the cpu constrain typical real-world improvements, e.g. internal bandwidth (i.e. you need many SIMD ops between each load/store to approach theoretic performance, i.e. applicable only to complex functions)
For a cpu that is to live 3-5 years mainly doing Lightroom (and similar tasks), estimating the weights to each of the 3 points above might lead to entirely different purchase decitions. 1) and 3) would suggest to upgrade other parts of the system (or live with the performance for now). 2) might suggest a cautious upgrade to Haswell, or more digging.
This would require an e.g. 4x4 testing matrix to be computed per application every time a new major (minor) revision of the software is released. I realize that this would increase the workload of testers comparing to doing only a single test of some (fixed) version of that application that can be added to a list of previously reviewed cpus using the same fixed version of the application. Perhaps this can be somewhat compensated by picking a few real-world, wide-spread and time-consuming applications (and not (semi-) synthetic benchmarks etc that seems to serve a different purpose, namely to analyze cpu performance somewhat isolated from software).
http://www.anandtech.com/show/7003/the-haswell-review-intel-core-i74770k-i54560k-tested/6
http://www.anandtech.com/show/7113/2013-macbook-air-core-i5-4250u-vs-core-i7-4650u/2
http://images.anandtech.com/graphs/graph7113/56102.png
http://www.sweclockers.com/recension/17016-intel-core-i7-4770k-och-i5-4670k-haswell/12
http://www.xbitlabs.com/articles/cpu/display/core-i7-4770k_9.html
http://www.michaeldeleon.net/lightroom-3-vs-lightroom-4-benchmarking-previews-and-export/
regards
k
edit:
While new (5.0) versions of Lightroom does seem to thread export of files (unlike recent older ones), if you start two jobs, those jobs seems to give higher system load (I did not time execution time, but assume that it will be lower). This just adds to the complexity...
So what if the Intel Haswell performs e.g. 8% better at the same clock than Ivy Bridge or 14% better than Sandy Bridge when performing a set of image processing tasks in Lightroom 3.0. Lightroom 5.0 (and 5.2RC) is out. We don't know when/if Adobe engineers switched intel compiler (-switches), or how much/early they are doing low-level intrinsics/assembler for a new architecture. We don't even know to what degree Adobe relies on external libraries to do the grunt work (Intels IPP?)
In order to have some insight into this, I'd suggest to profile Lightroom 3.0, 4.0 and 5.0 (possibly minor revisions as well) on the last 3-4 intel architectures. The task might be something like "import 100 raw files, display on color-profiled screen and apply default settings, export as jpeg". Now, the algorithms have changed over the years, so comparing absolute numbers across Lightroom versions might not make sense. But the relative scaling of different Intel architectures should tell us a lot about how aggressive Adobe are at explicitly compiling/optimizing for new architectures (how much time does it take from Intel release something new before Adobe can take advantage of it, and to what degree do they benefit from it).
As is, I know that Haswell introduce major improvements in integer SIMD (just the kind of thing that image processing might benefit from, AVX2 should double the theoretical throughput of integer SSE?), and smaller but still significant improvements to floating-point SIMD (FMA should double theoretical throughtput over AVX1 for common linear filtering tasks? for all I know, Lightroom might be heavily floating-point dependent). When the measured performance of old-ish versions of Lightroom does not increase by 2x, this could be due to any mix of the following:
1. The algorithm is not perfectly SIMD limited (inherently single-threaded parts, heavy memory/disk access). Increasing vectorized performance is unlikely to ever improve this algorithm much.
2. Adobe were focusing on other things than low-level optimization for cutting-edge hardware (or the hardware were not available to them at the time of compilation). They may or may not catch up with Haswell in newer versions.
3. The Intel improvements are theoretical, other parts of the cpu constrain typical real-world improvements, e.g. internal bandwidth (i.e. you need many SIMD ops between each load/store to approach theoretic performance, i.e. applicable only to complex functions)
For a cpu that is to live 3-5 years mainly doing Lightroom (and similar tasks), estimating the weights to each of the 3 points above might lead to entirely different purchase decitions. 1) and 3) would suggest to upgrade other parts of the system (or live with the performance for now). 2) might suggest a cautious upgrade to Haswell, or more digging.
This would require an e.g. 4x4 testing matrix to be computed per application every time a new major (minor) revision of the software is released. I realize that this would increase the workload of testers comparing to doing only a single test of some (fixed) version of that application that can be added to a list of previously reviewed cpus using the same fixed version of the application. Perhaps this can be somewhat compensated by picking a few real-world, wide-spread and time-consuming applications (and not (semi-) synthetic benchmarks etc that seems to serve a different purpose, namely to analyze cpu performance somewhat isolated from software).
http://www.anandtech.com/show/7003/the-haswell-review-intel-core-i74770k-i54560k-tested/6
http://www.anandtech.com/show/7113/2013-macbook-air-core-i5-4250u-vs-core-i7-4650u/2
http://images.anandtech.com/graphs/graph7113/56102.png
http://www.sweclockers.com/recension/17016-intel-core-i7-4770k-och-i5-4670k-haswell/12
http://www.xbitlabs.com/articles/cpu/display/core-i7-4770k_9.html
http://www.michaeldeleon.net/lightroom-3-vs-lightroom-4-benchmarking-previews-and-export/
regards
k
edit:
While new (5.0) versions of Lightroom does seem to thread export of files (unlike recent older ones), if you start two jobs, those jobs seems to give higher system load (I did not time execution time, but assume that it will be lower). This just adds to the complexity...
Last edited:
Facebook
Twitter