Database @ work

[Aimster]

At work we connect to a mainframe computer to run some queries off of a database system. One particular folder has 5,000,000 records and the database cannot complete this tasks. It just hangs....

I thought with today's technology we could easily write a query for 5,000,000 records, but I guess not.
They claim the mainframe computer is up-to-date with the newest technology.

5,000,000 records being no more than 10 lines each.

Is it the mainframe that is the problem or is the 1970s database program not up to the task no matter what kind of mainframe it is on.

 

[somethingsketchy]

Originally posted by: Aimster
At work we connect to a mainframe computer to run some queries off of a database system. One particular folder has 5,000,000 records and the database cannot complete this tasks. It just hangs....

I thought with today's technology we could easily write a query for 5,000,000 records, but I guess not.
They claim the mainframe computer is up-to-date with the newest technology.

5,000,000 records being no more than 10 lines each.

Is it the mainframe that is the problem or is the 1970s database program not up to the task no matter what kind of mainframe it is on.

It could be a lot of things. It could be the network traffic (if everyone are doing things they're not supposed to) that can create a bit of delay. Then there's the software itself. If it is poorly written then you'll have the "hang" you're experiencing.

Then again 50 million lines (give or take) is a lot of information for a 1970s database to process. Might be helpful to consider updating to a newer build of the database program (if it is even possible on many fronts - financially, logistically, etc).

 

[hans007]

when i was in college we replaced the 25 year old pa-risc server, with a brand new ...HP pa-risc server.

THe old one was 2 refridgerators in size, the new one was a 1U unit. THat said, it ran the exact same software with all the exact same bottlenecks.

If you want to make the database take advantage of the new server, you could figure out a way to extract all the data from the old database and put it in say a new sql one. Sure it might take a long time to port over but after that assuming you choose a good new database it would be pretty fast.

5 million records is still a lot of rows of data. I have worked on oracle 10 based systems before (which is fairly current though not state of the art) and going over say 200k records still takes some time to get the query results sometime (by some time i mean say a minute). I am not sure what you mean by "hangs"

 

[WildHorse]

I agree with somethingsketchy. It could be a lot of things.

What rdbms (database system) is it?

As you know, different systems are intended for different levels of usage, e.g.; MS Access for a person or small office, MySQL for a mid-sized corporation, Oracle for a big corporation.

Always remember what Mr. Natural said, "Get the right tool for the job!" You might have a misfit of rdbms for the type of usage you give it. Or maybe it doesn't accept modern SQL, or maybe a networking issue, or lots of other possibilities for your annoyance.

5 million records isn't large (by corporate standards) enough to give modern database systems any trouble. There have been times I've probably had more than that on my personal pc using MySQL.

 

[Paperdoc]

"1970's database" made me remember. I hope the db software is client / server based, not the original style that merely kept the data files on a central server. In that old system when you ran a query, the server merely transmitted all the records to the user's machine and all the query processing was done at your desk. The change to client / server designs that processed the entire query at the server and then transmitted the resulting report happened well after the'70's - in fact, more like the mid-80's - for desktop systems, and I'm not sure when for mainframe applications.

 

[WildHorse]

Aimster,

Here's something you or your IT people might check out as a possible alternative to your ancient RDBMS,
as suggested by iCyborg in the AnandTech Programming forum.


edit: the above link only seems to work when you're logged into AnandTech. Strange.

Here's what iCyborg had suggested in that other thread: SQLite