Script Database /w unforseen expectations

[Drakkon]

So I have this collection of videos that have external script files in xml formatting. About 100mb worth but changes every so often (added to, subtracted from). They are all housed on a separate streaming server off site thats only accessible via FTP. Now they are asking all the scripts to be search-able.

I only have PHP, MySQL, PERL to work with. I only have 50mb to work with. PHP doesn't have the FTP routines enabled (and they will not enable them for me) so thats my first challenge.

It seems like searches would be incredibly slow if i had to open each file, have a parser read through it, then regexp to match lines or something. O thought putting the files into MySQL would lead to faster searches, but then that would pass my 50mb barrier. Right?

Are there any other options?

 

[WannaFly]

hmm, sounds like an interesting situation, but the 50mb is def going to be a big barrier. I'd imagine you could connect to their FTP, download a file, process it into the DB, then move to the next. 100MB of XML *MIGHT* be less then 50mb in MySQL because of compression/binary/etc.

Can you run a remote process on the server that holds the xml files, that combines them into 1 xml file with only the data you need, then you can download that one file/

I agree with how slow it would be to process each file, you need to do some sort of batch import to Mysql.

Also important - how often are they updated? Randomly I assume.