Python Question: how to tell if a file is open?

[Childs]

Similar to doing an lsof. I want to tell if a file is done copying before I process it. The copying is external to the script. I originally checked the mod time, waited an interval, then checked the time again. A colleague pointed out that there could be a network outage, so the interval wouldnt be enough to detect if it finished. Of course he didnt know how to do it, but its a valid point. I cannot find a method in the standard library to do this. Any suggestions?

And C programmers, how would you do it in C?

 

[bersl2]

What is the nature of the copying of the file?

 

[notfred]

What OS, Filesystem, and how are the files being written?

Maybe you should grab the source code for lsof.

 

[Childs]

The copying is simply someone dropping a file in a folder. My script monitors the directory, and submits the file to our distributed rendering system. It works, based on getting the mod time, waiting 15 secs, then getting the mod time again and seeing if its changed. I'm just trying to solve the edge case where the network goes out or in the process of copying the file the script is somehow fooled into thinking the file is complete. Kinda unlikely, put possible. I can always increase the interval, but if theres a method that simply returns an int if the file is open/closed then that would make things easy.

OS is Mac OS X, but Linux support would be great. I was going to run through the source of lsof if I couldnt find a method, I just want to make sure I'm not missing something in the standard lib. I havent written in python in awhile, so I dont know if I'm forgetting something obvious. I tried to use fcntl to see if I could get an exclusive lock on file and do a bool test, but it didnt work, as null is returned. Not sure if a script running as a regular user even has access to this info.

 

[Templeton]

*I'm not a python programmer*
I'd think you should be able to move/rename the file, the operation should fail if another process is writing to it. (in

 

[Armitage]

why not make a system call to lsof with one of the popen varients and parse the output for what you need?

 

[Cooler]

//In C++ for wrinting to a text file.

ofstream a;
a.open("/mypath/mydoc.txt");

a<<"My text that I am adding.";

a.close();
// Reading a textfile
ifstream b;
b.open("/mypath/mydoc.txt");
string st;
while (b>> st){
// Process word
}

b.close();


// Will check if you cant open file for some reason
if (a.fail()) {

}

 

[Childs]

Originally posted by: Templeton
*I'm not a python programmer*
I'd think you should be able to move/rename the file, the operation should fail if another process is writing to it. (in

In unix-like systems it wont fail. The app that was writing the file would fail.

Originally posted by: Armitage
why not make a system call to lsof with one of the popen varients and parse the output for what you need?

The main problem is you'd need root access to view all open files. This needs to be run by a normal user who wont have root access. Plus, I just wanted to know if there was a lower level way to do it.

Originally posted by: Cooler
//In C++ for wrinting to a text file.

ofstream a;
a.open("/mypath/mydoc.txt");

a<<"My text that I am adding.";

a.close();
// Reading a textfile
ifstream b;
b.open("/mypath/mydoc.txt");
string st;
while (b>> st){
// Process word
}

b.close();


// Will check if you cant open file for some reason
if (a.fail()) {

}

I tried something similar, but what I found was the file methods are testing what you just opened. If you're opening a file in your script, you can tell if its open or closed or failed on open. But if you dont open the file you dont have your file descripter and you cant test a non-existent fh. If you define your file that very action opens the file, so your test is always going to say its open. Maybe I'm missing something. On unix you can always open a file, even if its already open by other apps.

Thanks anyways. I appreciate the help.

 

[Childs]

I wasn't passing an int to flock, which is why it didnt work the first time I tried it. Also, the option I specified wasnt a method of flock, as I treated it like a string. Assume the example below checks for the argv[1], and permissions are not an issue.