Fvcking "Man in the loop"

[Armitage]

Let's replace them all with computers

More of my file-parsing woes ... my process just died because some bozo fat-fingered 'september' as 'septimber', and of course, strptime has no clue what that means.

So why is somebody typing dates into a report that gets should be generated by whatever program generates the analysis in the first place?? What else have they fat-fingered in these reports?

I've ranted on this before, but it's particularly ironic, as I just spoke abotu these kinds of problems to the person responsible for creating these files last week. They "pride themselves on having a man-in-the-loop on all of these processes for quality control".

Sorry buddy, but you'd be much better off automating the process, and letting your "man-in-the-loop" supervise and check the results.

Oh well ... how many ways can I spell "september" :disgust:

 

[Armitage]

More joy ... they are randomly changing the format of the date field in the actual data. There goes my hope that at least the important bit was automatically generated, instead of typed in manually :disgust:

 

[drag]

Were are they inputting this data? Into a plain text file or something?

Can't you restrict what they put in the feilds, like having a drop down list or at least checking that it's spelled correctly before accepting it?

 

[kamper]

Wasn't it Armitage that said he would shoot himself if he had to program drop down lists? Or something like that?

 

[AFB]

Originally posted by: kamper
Wasn't it Armitage that said he would shoot himself if he had to program drop down lists? Or something like that?

(amdfanboy) Say you had to order the items in a drop down list. Performance isn't going to mean much

(Armitage) Well yea. If that's all I did in programming though, I think I'd rather dig ditches

WORD

 

[Armitage]

Originally posted by: kamper
Wasn't it Armitage that said he would shoot himself if he had to program drop down lists? Or something like that?

Yea, that was me. And let me tell you, writing file parsers isn't to far down that list either

The problem is, I have no control whatsoever over how these files are generated. As far as I am concerned, they appear by magic on and FTP server, and I get to use them.

When I complain, they smile and nod, make agreeable noises, but keep on using their same tools and procedures that were developed in ... no kidding ... the 1960s.

Well, actually, this process must be newer then the 60's, as the files come across in Excel format ... grrr ... of all the things they should have kept, ASCII files are one!

 

[Childs]

Why not parse the fields and run sanity checks on the data, then write the records to another file if you can't decipher it?

 

[Armitage]

Originally posted by: Childs
Why not parse the fields and run sanity checks on the data, then write the records to another file if you can't decipher it?

Yea, that's basically what will happen. If the parser fails, it will log the failure and send me an email so I can go take a look at it. Thing is there are so many files, updated so frequently, that this could turn into a full time job cleaning up after these guys.

 

[Childs]

You could analyze each field by character, noting which characters are in the field, then compare those characters to a string representing a particular month. Then guestimate based on which characters are present which month it is (case), and rewrite it. I'd also log that record in case there are some discrepancies.

I did something like this awhile ago using C. By looking at the characters in each string for each month, it looks like the string could be misspelled by up to three characters, but as long as the first and last character are correct it should be fairly accurate. I would probably compare each character to all the months, and drop months which fails the match. If there are no matches, then go to some arbitrary character position and compare again. After three characters you should have a match. Overhead should be minimal.

 

[drag]

Originally posted by: Childs
Why not parse the fields and run sanity checks on the data, then write the records to another file if you can't decipher it?

That's what I ment...

Seems like a basic programming stuff for me, you always check your inputs. I'd hate to have a program crash because somebody hit a , instead of a .

 

[stnicralisk]

Originally posted by: Armitage
abotu

Damn fat fingered bafoons! Replace everyones jobs with computers!

 

[Armitage]

Originally posted by: drag

Originally posted by: Childs
Why not parse the fields and run sanity checks on the data, then write the records to another file if you can't decipher it?

That's what I ment...

Seems like a basic programming stuff for me, you always check your inputs. I'd hate to have a program crash because somebody hit a , instead of a .

1. These guys are making it harder then it needs to be. I'm not a programmer, and don't want to be.

2. It's not so much these obvious, easily caught errors that I'm concerned about, but what it indicates about the overall process that produces the data. "septimber" isn't that big a deal ... it throws an error, I fix it. PITA, hence the rant, but no big deal. But if they are manually typing in all the data, as seems to be the case, everything is suspect, and there is a big deal between 16256 and 16526, but much harder to trap for. Transposed numbers are another type of error I've found in a similar file.

 

[drag]

Originally posted by: Armitage

Originally posted by: drag

Originally posted by: Childs
Why not parse the fields and run sanity checks on the data, then write the records to another file if you can't decipher it?

That's what I ment...

Seems like a basic programming stuff for me, you always check your inputs. I'd hate to have a program crash because somebody hit a , instead of a .

1. These guys are making it harder then it needs to be. I'm not a programmer, and don't want to be.

Well you are! Congrats on the promotion!

2. It's not so much these obvious, easily caught errors that I'm concerned about, but what it indicates about the overall process that produces the data. "septimber" isn't that big a deal ... it throws an error, I fix it. PITA, hence the rant, but no big deal. But if they are manually typing in all the data, as seems to be the case, everything is suspect, and there is a big deal between 16256 and 16526, but much harder to trap for. Transposed numbers are another type of error I've found in a similar file.

Your right about that. Every single time you think you make something idiot proof, you find out quickly that the world just produces better idiots in response.