How do I do this?

[Red Squirrel]

I have a python script that fetches data from a serial device, it is basically alarm points. I want to store these somewhere and update it often like once a second.

I then have a custom app that needs to be able to read these. I tried just writing it to a text file on a ram drive, but the issue I'm having is with both reading and writing at the same time to the same file, I keep getting "sh: permission denied" errors in the app that tries to read.

Is there a better way of doing this? Originally I had the app doing a serial connection for each alarm point but given it takes a few seconds to get connected, it adds up fast. I rather be able to fetch all values at once from an external script then just have my program poll the values from a file or something every second.

Also is there a way to debug permission denied errors? Even if the script that's writing the file is not running I STILL get those damn errors. Everything is running as root, so there is no reason for this. Something else must be going wonky but I have no idea what.


Edit: figured out the permission denied stuff. Really stupid actually, I forgot to put "cat" for the file it was reading and passing through grep/sed. So instead it was trying to execute it. Still curious about a better solution though as having two independant processes reading and writing to the same file seems like a bad idea. But now it's working fine...

[jimmybgood9]

Maybe try a named pipe. Reader app waits on something coming through the pipe. Script fetches alarm points and dumps them down the pipes. Reader app gets alarm points and waits again. Script waits a second and fetches more alarm points.

Just a thought

[mv2devnull]

A bit heavier is to use a SQL database, say mysql. Script updates table, app queries table. An update should be a single transaction, or else the reader could see a halfway changed table.

[Red Squirrel]

Yeah SQL is overkill. Keep in mind too both apps are completely independant of each other, though I'm starting to wonder if I should just impliment the serial read into the app, but serial comm with C++ is a pain in the butt. It's really flaky.

Actually, is there a way to create a /proc type file? Wonder if I could just do something like /proc/alarms/1 etc and have virtual files with the values.

[Nothinman]

Ideally I would think the serial I/O should either be in the main app or a library to which it's linked.

/proc files are created dynamically within the kernel so you could have to create a new driver or amend the serial one which will most likely just have you end up with a new file that does almost the same thing as the device file in /dev. You really don't want to go there for multiple reasons. Primarily because kernel development is very different from userland development, secondly /sys is where that stuff should go now as /proc was deprecated for this kind of stuff a long time ago.

You don't need a full RDBMS to maintain a small SQL DB, you could use a SQLite file.

jimmybgood9's idea isn't bad either. A named pipe could work or you could have your reader script print the data and then have your main app execute the reader script and read the data from its stdout.

[Fallen Kell]

Originally Posted by Red Squirrel

Still curious about a better solution though as having two independant processes reading and writing to the same file seems like a bad idea. But now it's working fine...

If the second process is only ever reading, there is no problem. If both were trying to read and write, well, you could run into issues depending on how you were modifying the file. But one process writing a file and another periodically checking/reading that file is a very common practice.

A named pipe might also work in this case, however, I am unsure if you would be able to use the pipe the way you currently do. The pipe I believe is truly a FIFO queue. So if your one application writes to it 4 times, there are 4 separate writes in the pipe (it doesn't clear the previous data).

[mv2devnull]

Originally Posted by Nothinman

You don't need a full RDBMS to maintain a small SQL DB, you could use a SQLite file.

I thought that SQLite does not allow simultaneous sessions; without them it does not need to implement locking. If both the script and app do have the DB open at the same time, then SQLite does not work.

Does the pipe actually need a name? How about a simple:

script | app

where the script (Python) writes to sys.stdout and the app reads std::cin (like any std::istream).

[Red Squirrel]

Originally Posted by Fallen Kell

If the second process is only ever reading, there is no problem. If both were trying to read and write, well, you could run into issues depending on how you were modifying the file. But one process writing a file and another periodically checking/reading that file is a very common practice.

A named pipe might also work in this case, however, I am unsure if you would be able to use the pipe the way you currently do. The pipe I believe is truly a FIFO queue. So if your one application writes to it 4 times, there are 4 separate writes in the pipe (it doesn't clear the previous data).

That's good to know, so maybe I'll just keep it how it is then as after finding that silly mistake of mine now it all works ok.

The reader app is an environmental/server monitoring app, so it simply loops through all the alarm points and polls each value. Monitor values are basically the result of a Linux command. Most are cat/sed/grep of a command's output to get the value, such as disk space, memory and so on. This particular command was reading a text file generated by the arduino reader script which I really did not want to have to integrate as on it's own it has nothing to do with the monitor app. So I'm glad this ended up working out ok. I have it setup where the script is just fired up at startup and keeps looping so the text file on the ram drive is always up to date, and my monitoring app simply monitors each line's values, including the date it was last modified, so if the script that updates it fails, I get an alarm for that too. Got all my bases covered now. That error was just screwing with my mind as I have a similar monitor for my hvac system and it works fine, but then turned out I forgot to put cat lol. so instead of executing cat, it was trying to execute the text file.