pc build for very large, detailed GIS maps

[bookman3]

Hi all

I need to build/purchase a pc specifically for designing national maps with detail down to 5-digit zips. My current pc is a 'high-powered' dell workstation, but is extremely slow for the task. Boss would want to purchase, though if it saves enough money I probably could convince him to let me build.

Is there a hardware (or software) setup that would allow me to speed up this process? SAS or SSD based raid? 64bit (more RAM), multiple processor support, or CUDA/GPU support (ie.Manifold)?

Ideally these national maps would draw within 2 minutes.

_____________________

The situation:

I'm currently using ArcView Single-use licence on a Dell Precision T5400 and am having speed issues. I use GIS in a very specific way and need it to be very quick.

I use it to map the results of logistic regression models that rank 5-digit zip codes based on how likely a resident of each ZIP is to be interested in each of our clients. To do this I join our data (ZIP5 and a score 1-9) with a ZIP5 shapefile. I need to create state maps and national maps, both using 5-digit zips.
The state maps draw slowly, but bearably so. The national maps are way too slow. It takes 20+ minutes for each map to draw and 45min to many hours for them to print. Often with Arc crashing in the process. This means that if there is a conflict in labeling, or some other aspect, I need to click pause, wait for Arc to register & respond to my click (5+min), make the change, wait for drawing (20+min), check if fixed, if not repeat.

 

[sgrinavi]

There's many ways to configure your current work station, do you have specifics on the current configuration?

EDIT

Have you looked at the system requirements? woah....

http://support.esri.com/index.cfm?f...tID=25&pvName=3.3&PID=25&versionID=35&PVID=96

 

[vshah]

Hardware Requirements

Memory/RAM:

24 MB Required / 64 MB Recommended

NOTE:
The requirements to run ArcView GIS fall within the MHz and RAM system requirements which Windows XP requires.


Best Performance Configuration

Screen Resolution:
1024 x 768 recommended or higher at Normal size (96dpi)
http://support.esri.com/index.cfm?f...tID=25&pvName=3.3&PID=25&versionID=35&PVID=96

is that what you're seeing? i'm confused...

 

[lxskllr]

When it's churning a map out, are you getting heavy hard drive activity, pegged CPU, or what? I'm not familiar with that software, but you may be able to pin down the problem area that way.

 

[sgrinavi]

is that what you're seeing? i'm confused...

Ya, that's what I'm seeing too. That's why I was asking about his current system config. Looks like it is not a naitive windows app and might be better off running a different OS.

If it were my ass on the line I would be calling the software manfacturer.....

 

[bookman3]

hey guys, just got back from an extended lunch.

Here's the link to the system reqs. I'm currently using version 9.3.1
http://wikis.esri.com/wiki/display/ag93bsr/ArcGIS+Desktop

The workstation has an Intel Xeon E5405, 2gb of RAM, Quadro NVS 290 gfx, WD 80gb WD800JD HD and is running winXP 32bit. The program uses "sequential process threads", so it doesn't take advantage of multi-core processors very well (http://support.esri.com/index.cfm?fa=knowledgebase.techArticles.articleShow&d=31903).

I don't think its common for people to make maps with ArcGIS at the national scale using data as small/detailed as 5-digit zips, but it is very useful in this circumstance. Since reads & processes data sequentially, my thought was that I should have as fast a HD & processor speed as possible.. not sure though.

 

[bookman3]

Also, CPU usage jumps around on one of the cores (between 60-100&#37 when a map is drawing, the other 3 remain relatively idle.

I'll look around also, but what prog should I use to check HD activity?

 

[lxskllr]

Also, CPU usage jumps around on one of the cores (between 60-100%) when a map is drawing, the other 3 remain relatively idle.

I'll look around also, but what prog should I use to check HD activity?

I'd just look at the HD LED. If that's solid on while the map is rendering, it would indicate excessive hard drive use. Not scientific, but it would help nail the problem down.

 

[sgrinavi]

Well, without a doubt an SSD and a couple GB of RAM would punch things up. I would even consider an upgrade to win 7/64. Have you tried calling the manufacturer?

 

[TheKub]

Are the maps stored on your workstation or on a server? In my experience in supporting GIS users getting gig ethernet was far more of a performance boost over upgrading hardware.

 

[bookman3]

I store the maps & data on the workstation. We have gig ethernet, but storing on the network was still very slow.

I may be able to convince our dba/tech-support person to allow a Win7/64 pc (w/ SSD) on the network, but we did finally just upgrade the server to 2003 & office from 03 to 07 over the holidays. We're still using XP/32 & SPSS 14.0 (rel date: Nov 2005, current vers: PSAW 18.0).

My thought has been that Win7/64 with 8 or 12 gb of ram & an SSD (or 2 or 3 in a RAID) would make this possible. Or maybe all of this with a CUDA/GPU card set up to do parallel processing using a different GIS program that can take advantage of it (ie Manifold).

If I can gather enough supporting evidence that it would allow the maps to be drawn in min/sec instead of an hour, I could get the boss to approve it, despite resistance from elsewhere.


(Up until a year ago, we were telling our clients (and employees) that they must use internet explorer cause we weren't willing to test/support for firefox or chrome. The addition of some younger employees (including myself) assisted in getting rid of that kinda silly policy. I'm sure all of this is more common than I've assumed in the past, but coooomme oon...)

 

[heyheybooboo]

I don't want to say anything bad about ESRI ArcView but ...

Manifold System Release 8

 

[Knavish]

If you think the disk is the bottleneck and the program only takes 2.4 GB on disk, why not try some 3rd party ram disk software (random example: http://members.fortunecity.com/ramdisk/RAMDisk/ramdriv002.htm). You could copy / install the program to a temp ram disk & see if it speeds up greatly.

I suppose you'd need a 64-bit machine with 6-8GB ram minimum to test this...

Do you have access to a more capable machine (like your personal computer) that you could use to test this?

 

[bookman3]

k... It appears there is little HD activity while the maps is drawing, but the CPU usage caps out at a steady 25%. Its a 4 core CPU and the activity is split across the four processors, but never goes above 25%.

My personal comp is an ASUS p5q w/ C2D E6300, 2gb ram & an X-25m g2 80g SSD and also a new 2nd tier 15in MBPro. ESRI's licensing is such a PITA that I can't easily try it out on my comp at home. When i've tried w/ a diff license I couldnt get it to work in 64bit, only 32bit. I could buy the extra RAM, install w7/64 and test Manifold tho..

 

[theevilsharpie]

If I can gather enough supporting evidence that it would allow the maps to be drawn in min/sec instead of an hour, I could get the boss to approve it, despite resistance from elsewhere.

Windows' built-in performance monitoring tools will tell you pretty much everything (save for 3D performance) you need to know. Here's a quick overview of how to use them.

http://technet.microsoft.com/en-us/magazine/2008.08.pulse.aspx?pr=blog

Based on your posts, the processor is the most likely bottleneck.

 

[PsiStar]

First, find out if a multi-threaded version of ArcView exists and get that!

What you have is definitely single threaded & is why only 25% CPU usage ever appears. If a multi-threaded version does not exist ... well my 1st reaction is to find different software! But in lieu of that, you need the fastest single CPU (single core) processors that can be found. I will guess that is the i7 975.

Yes, this is still 4 core, but I am assuming that you ArcView actually does have a multi-threaded version. This will fix 1 bottle neck.

Thinking positively about the multi-threaded, get a SSD to be your work drive. The reasoning being that once the CPU bottle neck is fixed you will then be throttled down to the speed of the HDD ... so get a SSD.

 

[mpo]

Before looking at a hardware solution, how is your data organized? Does everything have the same projection?

Are you storing your data? In plain shapefiles? As feature classes in a file/personal geodatabase? How big are the files?

Do all of your shapefiles have spatial indexes? Are the tables you are joining indexed?

(You might want to post your question on the ESRI user forum. They're really helpful.)