- Mar 8, 2003
- 38,416
- 4
- 0
Greetings,
Was fiddling with methods to communicate between a Java client and a PHP server.
Lets say that I am dealing with a very large array of structs containing plain text. Each one of the elements are independent of one another.
Currently, that array of structs is a single large XML file.
On the server, I am using PHP's XML Pull parser to read data as a stream from the client and perform operations on each line of data as it is read, rather than waiting for the entire file to be received and load the entire thing into memory at once. Once it is finished processing the data, I send the results down to the client in a similar XML file. The task takes a little while to run, so I write the resulting elements to the XML stream as they are finished.
On the client side, I am doing something similar with SJSXP (Sun Java Streaming XML Parser). I am reading an element from the array from the stream and performing an operation on it without waiting for the entire file and loading it all into memory. Another problem is that the client does not have nearly as much ram as the server, so it does not make sense to load a 100mb XML file into memory on the client.
I am not sure how many results will be needed at the beginning as I am processing the requests on the fly. I know it is more efficient to send it all at once, but I am doing buffering at both ends even with the writes to the buffer only occurring on one element at a time.
I have not dealt with JSON very much, but it seems to be able to do what I am doing with my custom XML files in a tiny fraction of the amount of code. The only problem is, I am not sure how efficient it is for this kind of task.
So, my question would be, in addition to any other insight that could be provided:
Would it be more efficient to keep my current XML stream parsing or switch to JSON? Seems like I would need to establish a new connection for a transaction each time I wanted to send an element as it is completed, unless I wait for everything to be finished and send it down as a JSON array, which would cause lag for the client. But, there could be something similar to SJSXP that I have not discovered yet that will allow me to keep the everything open as results are returned and treat it as a stream.
Thanks
Was fiddling with methods to communicate between a Java client and a PHP server.
Lets say that I am dealing with a very large array of structs containing plain text. Each one of the elements are independent of one another.
Currently, that array of structs is a single large XML file.
On the server, I am using PHP's XML Pull parser to read data as a stream from the client and perform operations on each line of data as it is read, rather than waiting for the entire file to be received and load the entire thing into memory at once. Once it is finished processing the data, I send the results down to the client in a similar XML file. The task takes a little while to run, so I write the resulting elements to the XML stream as they are finished.
On the client side, I am doing something similar with SJSXP (Sun Java Streaming XML Parser). I am reading an element from the array from the stream and performing an operation on it without waiting for the entire file and loading it all into memory. Another problem is that the client does not have nearly as much ram as the server, so it does not make sense to load a 100mb XML file into memory on the client.
I am not sure how many results will be needed at the beginning as I am processing the requests on the fly. I know it is more efficient to send it all at once, but I am doing buffering at both ends even with the writes to the buffer only occurring on one element at a time.
I have not dealt with JSON very much, but it seems to be able to do what I am doing with my custom XML files in a tiny fraction of the amount of code. The only problem is, I am not sure how efficient it is for this kind of task.
So, my question would be, in addition to any other insight that could be provided:
Would it be more efficient to keep my current XML stream parsing or switch to JSON? Seems like I would need to establish a new connection for a transaction each time I wanted to send an element as it is completed, unless I wait for everything to be finished and send it down as a JSON array, which would cause lag for the client. But, there could be something similar to SJSXP that I have not discovered yet that will allow me to keep the everything open as results are returned and treat it as a stream.
Thanks
)
Facebook
Twitter