Move / Rename

[txrandom]

I have a php script that downloads a jpg to a temporary directory. The jpg is then converted to a png using ImageMagick. After completing the conversion, I use the php script to move (using rename()) the png to a live directory. I have another script that takes the newest image from the live directory and spits out the contents.

Originally I didn't have a temporary folder, and I ran into a major problem. The conversion process created the PNG file, but didn't actually finish writing all the contents of the png until half a second or so later. This meant the other script was occasionally reading an incomplete or corrupt image.

Will moving/renaming an image still leave the possibility of the php script reading an "incompletely-moved" image? From my understanding of computers, all that happens in a rename or move is that some paths between trees are moved around. The file itself is never even touched correct?

If we were talking about copying the png from a tmp directory to a live directory, we could potentially have that problem, correct? Although can a copied file even be accessed before it's finished copying?

 

[Nothinman]

Yes, a rename just changes the directory entry, it doesn't touch the file data itself.

Although I don't know why your script was seeing the incomplete PNG file unless it opened the file before the ImageMagick script finished, even if the data wasn't flushed to disk it still should've been visible from the write cache.

 

[txrandom]

Originally posted by: Nothinman
Yes, a rename just changes the directory entry, it doesn't touch the file data itself.

Although I don't know why your script was seeing the incomplete PNG file unless it opened the file before the ImageMagick script finished, even if the data wasn't flushed to disk it still should've been visible from the write cache.

I think ImageMagick creates the file before it's actually done converting it.

 

[Nothinman]

I think ImageMagick creates the file before it's actually done converting it.

Which is fine, so I guess your other script scans the directory periodically and would start on a file that wasn't done yet?

 

[txrandom]

Originally posted by: Nothinman

I think ImageMagick creates the file before it's actually done converting it.

Which is fine, so I guess your other script scans the directory periodically and would start on a file that wasn't done yet?

Yep, I think that's exactly what was happening. I did some more testing and the error even occurs when were using a completely static image. I believe it's just a problem with the devices now.

 

[Scarpozzi]

See if you can put a delay of some sort in your script. It may not totally solve the issue, but it will at least decrease the frequency of it.

 

[Nothinman]

Or since you wrote both scripts you could lock the file in the first and test for the lock in the second retrying until it succeeds.

 

[mozartrules]

Originally posted by: Nothinman
Yes, a rename just changes the directory entry, it doesn't touch the file data itself.

That does assume that the source and destination is in the same filesystem. This is not a given if you use a temporary folder.

 

[DaiShan]

Originally posted by: Nothinman
Or since you wrote both scripts you could lock the file in the first and test for the lock in the second retrying until it succeeds.

There is no need for this. As has already been stated, a rename (mv) is atomic. The problem you are describing is fairly common with watch directory type applications, and the simplest solution is to rename or mv the file.