SR28 How-to Document

I recently revised my how-to-download-the-USDA Standard Reference Food and Nutrition database for SR28. You can find the SR24 still on the site but the front page link now points at the current version.


Terrorists are in the news as is their wont. This has prompted a dither over what to do.

We need to keep doing what we have been doing and do it better. Since 9/11, our freedom continues to be eroded. This is just what the terrorists want.

My recommendation is that we continue to allow the NSA to monitor phone connections. And allow the encryption of phone content. A backdoor for the government is a backdoor for everyone as the government has demonstrated its lack of ability in keeping secrets.

We should continue the debate as to whether the NSA should share the data it collects with the FBI and others. I lean to “no sharing” of data without a court order. The NSA should be free to share findings of specific threats with other agencies.

Too Big To Fail?

Cleaning out my traveling bag, I found a clipping from Redmond Magazine, the Independent Voice of the Microsoft Community. The Editor, Jeffrey Schwartz comments in the November issue on large companies breaking up. This reminds me of a song that goes through my head from time to time (i.e. a couple of times per week). If you are an older person and have never heard it, I hope that you have saved heavily in non-debt, non-equity investments. A change is coming. If you are a younger person, keep going. You will prevail.
The fact of the matter is that too big to fail is too damn big. The anti-trust laws and the regulators are supposed to prevent this from happening. This is just additional evidence that the present and recent administrations are incompetent and, perhaps, corrupt.
But I digress. While Mr. Schartz’s astute editorial mentions HP, Symantec, and and Microsoft, IBM, and many large banks are also in this category. Please stand by. Adjust your set if you think it will help.

What’s the diff?

My problem was that my revised version of the Java program did not work as expected. While I believed that I had changed nothing but comments, the results were not the same. What had I changed?

“Oh, that’s easy” says I, “I’ll just diff the files and see what changed”. Well according to diff, EVERYTHING had changed. I used the regular options like -w, and -E, and -d, and -B and still no improvement.

“Duh”, he says, “it’s the encoding”. What is the diff option to compare two texts with different encoding? The short answer seems to be that there is no option.

But where there’s a will, there’s a way, particularly with Linux (and Unix). In Linux use the iconv command to do the conversion and pipe the output to diff for the compare.

If you do not know iconv, the short form is:

iconv -f from-file-encoding -t to-file-encoding from-file-name

Output to stdout.

Pipe it to diff and tell diff to pick up the input from the pipe with a dash – .

If you do not know the encoding of the files, use the file command to show you the encoding.

To find out if the encoding is supported (and spelled correctly), use

iconv -l

for a list of supported encodings.

The answer to the original question looks like:

iconv -f ISO-8859-1 -t UTF-8 ../x3-2.14/x3navaldv0.2.14/java/bateau/ | diff -w - java/bateau/

I had changed only comments. Compiler difference? Compiler option difference?

FLTK – Fast Light Tool Kit

While working on an earlier project, I ran across a very good, portable window/widget tool kit. It goes by the name of FLTK (pronounced full-tick). A good tutorials exist here (note especially the “label pitfall”) and here which will teach you the basics. There are example programs in the full install found at the FLTK website along with an excellent programming manual. Documentation for previous releases is available. Ubuntu 10.4 LTS comes with FLTK 1.1.0.
When I began a new project, I decided to self-educate and used the second, pdf, tutorial above. I quickly worked through the examples until I got to 16. Handling mouse events part2 for the follow-the-mouse drawing program where I ran into a problem with the offscreenbuffer functions. I searched a bit for solutions. After reading more tutorials and more manuals, I discovered that the problem is that the example is omits several required includes. <stdlib.h> and <stdio.h>  are easy. A bit more difficult is #INCLUDE <FL/x.H>. This include provides environmental information for X11, in particular the definition for FL_Offscreen. These details are covered in Appendix F Operating System issues and briefly at “Drawing Things with FLTK” in the later on-line versions of the programming manual. But x.H may not be apparent at first glance. A corresponding win32.H also exists but including x.H will pick up the correct info if your environment variables are set.
With those three additions, the example 16 program compiles and runs.

Hacker Attire

Hacker in balaclava and black sweater using laptop
Hacker hacking
I was taking some “security training” for work. When they referred to “hackers”, the training showed “hackers” dressed in dark clothes and balaclava’s. I had seen this with other web sites. Since it is on the web and pervasive, it must be they way hackers really dress. I wondered why so I e-mailed my son who has degree in computer science and often is well informed on technical topics.

His response:

This is a carryover from the days before surface mount technology, when computer hacking required the use of a soldering iron.

The jumpsuit protected the “hacker’s” body from stray molten solder. The balaclava, which you’ll note covers the nose and mouth, protected the individual’s lungs from harmful fumes that can be released during soldering, including lead, arsenic, organobromides and resin acid particulates. Some of these gases can also cause contact dermatitis, a risk that is also abated by the jumpsuit.

Of course, modern-day hackers can simply exploit operating system design flaws to perform their work. The dress is now ceremonial, though its use is required in the EU (some laws are slow to change!)

Hope this helps…

lex and yacc Continued

Follow the link for an excellent tutorial ref1 on lex and yacc with some sample code. I will refer to this as ref1 in comments below.

Another lex and yacc tutorial with sample code . This post is based substantially on this second tutorial and the examples in the sample code.

A number of examples are provided. For each example, a script, say example1.compile, is present after you gunzip and tar x-tract.
chmod 755 the *.compile files to make them executable.
cat or edit each example, example4.l is the lex file and example4.y is the yacc file.

I have never been a fan of exercise, either in the gym or on paper. Give me a real problem or a real event to contest any day. But exercise is sometimes a necessary preparation for building strength and agility. Recalling Harvey Wagner and the “Mind Expanding Exercises” in his textbook Principles of Operations Research let us call these:

Stretching Exercises

Note: Do all of the examples. Stretching exercises extend but do not replace the tutorial.

0. Download the sample code. Gunzip and expand it. Mark the executable *.compile scripts and execute the example1.compile script. Resolve any knowledge gaps.

1. Build example2 and test it. Now enhance it so that it prints “MIXED” if the input is alpha-numeric, “NUMIXED” if the input is alpha-numeric with leading digit, “NUMBER” if all digits, “TEXT” if all letters and if the input is EXIT, Return 0.
Did you get a message like ex2.l:9: warning, rule cannot be matched on the lex execution? Why might it occur? How can you fix it if it does?

2. Build example4 and test it. What usability problems would you fix? Suggest 3 usability or reliability improvements. Create a version that implements the improvements.

3. Now create a version of the example4.compile script that includes the yacc “-rall” switch. Execute it and examine the y.output file. Do you think that the “-rall” switch aids understanding? Debugging? Checking? Testing? Designing?

4. Review the tutorial at ref1. Repeat exercise 3.

5. Create a word counting program using lex (flex). Nobody expects the Inquistion. Compile and test it.

6. Run example4.compile. Examine
rm y.output, lex.yy.c, and Run example1.compile again. Which files are created? Make note of the size(s).
mv or cp lex.yy.c to preserve it.
Run example4.compile again. Which files are different?
diff the preserved version of lex.yy.c and the just created one. Are they more or less different than you expected?

7. Run example4. Use the following inputs (from the tutorial)
heat on
heat off
target temperature 22

What were the responses?

Try “heat foo” – what is the result?
Try “target temp 99” – what is the result?

Rate the generated program on a robustness scale of 0-3.
Rate the generated program on a user friendliness scale of 0-3.
Explain how this experience will influence your future program designs.
Explain how this experience will influence your future program testing.

8. Between earlier examples and example4, can you explain what changed that allowed the removal of -ll (libfl if flex how can that be?) from the link specification?
What did yacc provide that allowed this to happen?
In your intermediate files, can you find the line number?

9. diff example4.l and example5.l
diff examle4.y and example5.y
Demonstrate your understanding of yytext and yylval by explaining the origin of their content and when it is created.
Further explain yytext and yylval as “c” variables in the generated program.

10. diff example4.y and example6.y
Explain your understanding of the origin of variable $2
Explain your understanding of the origin of variable $3
Explain why the specifics of $1 $2 $3 etc. are not that important.
Explain why the shared usage of $1 $2 $3 is important to the shared, generated code.

10. Explain the lexyacc efficency trade-off on the choices made for HEAT ON and HEAT OFF.

11. Run example5.compile and provide inputs from 7 above.
Describe your user experience:
a. Program correct?
b. Program robustness, security, reliability, resistance to failure.
c. Program presentation/beauty/ease-of-use/understandability
d. Program efficiency.

12. As a user, rank the factors a-d above. Check with your associates. Ask visitors to your home to evaluate. Ask strangers to evalute the list.

13. Create a version of example5 that is more user friendly. Consider recovery from errors, ease of use, lack of training, carelessness.

Now that you are warmed up, Wikipedia [Bison] and ref1 present calculator programmes. Build each, test and evaluate, share a blog post that compares and contrasts the available examples.

lex and yacc

If you lift the lid on the engine compartment of your car (bonnet to English-speakers, hood to Americans), you see a complex arrangement of TUBES, wires, boxes, containers, and mysterious objects. Worse, they may be concealed by covers, bundles and conduits. If you have a friend or perhaps classroom instruction, with the help of a readable manual, you can find your way and check your fluids, look for damage, identify the source of a mysterious new noise.

Should you decide that there is a problem that you can fix yourself, you will go to your tool box to choose the right wrench (spanner) or screwdriver to fix the problem.

Linux is not different. And the tool box is vast.

Or perhaps you wish to make something new and different. Linux (and Unix, and all that goes before) provides a huge tool box (the USPTO pretending to not know (“I am not knowing” (has this been outsourced too?)). The great advantage of the Linux distributions is that they are free (as in freedom) and free or inexpensive (in cost). When you receive benefit, give back.

In my home tool box, I have a basin wrench . Cost per use $5. It allows one to reach into the inaccessable space between the wall and the basin to loosen or tighten the nut that holds the faucet to the basin or the supply-line to the faucet.

LEX and YACC are just such tools. Like many tools in the Unix/linux world, the have a specific application. And they are absolutely genius in what they do.

“Proper tool for the proper job”. Each tool does one thing very well. Something else? That is why there is | (pipe)

If you are not trying to create a programme or solve specific problem, you may already know every thing you need to know.

lex – a tool to generate (in conjunction with yacc) C code for a lexical scanner
yacc – a tool to generate (in conjunction with lex) C code for a grammer interpreter.

In naive terms, lex – what do I see, yacc – what do I do with it.

Think about this problem: You are trying to create program that “understands” specific input words and is able to act on them.
lex is the tool that can do this. The “shorthand” description to lex and the C compiler can create a program for Linux that scans the input stream for specific values, and generates specific results.

If the inputs are more complicated, you may need to have the program interpret as an example “what to do” “thing that it is done to”. This is where yacc come in.

To add to the confusion, lex is the classic Unix program, flex is the Linux equivalent. Mostly compatable but subtly different.

Similarly, yacc is a Unix program, bison is the linux equivalent. Fortunately, flex and bison have compatability so previous knowledge and program source is not rendered valueless.

More in a future post.

Error Errors

Error message:Bad Opaque
Error Message 1

I got this error message box again. Can anyone tell me what it means?

I actually sort of know what it means, where it originates, and why. But it is a good example of bad programming practice and poor program development management that an end user sees a message like this.

What is wrong with the message? “Bad Opaque”. Yep. This message is worse than opaque, it is a black hole that sucks energy into oblivion. “Bad Juju” (not to be confused with juju) would be just as appropriate and more comprehensible. Where did this message come from? The title on the box provides a clue “Failed to enter room”. I was in a “virtual room” a while ago but the meeting is over and I have closed all of the visible windows. The message box is “on top” and will will not release focus. So, even though it is NOT OK, I click OK because that is the only thing to do.
Some time later, I discover an all-gray window with a title “Failed to connect”. Yep. The programmer clearly failed to connect the “Bad Opaque” message with the application. At least I guess they should be connected as the 3 windows appear to occur around the same time among the 10-12 windows I typically have open on my desktop with no indication on the task bar or system tray.
So my interpretation of this, which may or may not be accurate, is that the meeting I was in has ended, the window-less program that makes the connection tried to re-enter the now closed virtual room. Failing that in an unexpected way pops the “Bad Opaque”.
Somehow the connector or the room-enterer did not get the message that the session was over.  A bit more transparency and better error messages would be helpful.

Some Microsoft products have gotten better about making it clear what the error is and what the consequences of “Cancel”, “End”, “OK”, etc are. But not everybody there has gotten the memo yet, or if they have they have not read it and taken it to heart.

Picture of: Uncorrectable Error has occured UpdateHandel:handle not allocated Press Enter to Abort
I like this one too.  It certainly gets your atten- tion.  I comp- lained about it to the system administrator for several years.  I am sure that it is no longer an issue because that account moved to Google Mail eliminating the application that was uncorrectable.  At least this message is clear about what it means and where it comes from.  The problem with this one is that while I may not be able to correct the error, the programmer who failed to anticipate the condition but was able to provide an error message could correct the code to make sure that the LookupHandle was allocated or perhaps provide a suggested corrective action to the user.  At the very least, the programmer could suggest the external cause, if that were the case, so that external corrective action could be taken, preferably before but in any case after “Abort”-ing the application.

But my all-time favorite is this one.

Windows message box Cannot quit Microsoft Office
Yes you can!

Typical of Microsoft’s lack of quality control, due to some unexplained condition, you fall into a routine where the debugging software has been left engaged. The good news is that you can use Open Office on Linux to eliminate this message, Microsoft Office, AND best-of-all eliminate Windows.

Microsoft Excel – interesting feature Move Copy Worksheet

If you attempt to move or copy an Excel worksheet from a Excel 2007 or later workbook to an Excel 2003 workbook it will fail. The error message displayed does not give you any hint about the incompatibility and suggests cut and paste. You can complete the copy if you make the target workbook the same level as the sending workbook.
Do a Save As of the target worksheet to the new format. Close it. Re-open the just saved version to complete the conversion. The Move or Copy Worksheet function should then work properly.
This presumes that you actually found the Move Copy Worksheet function now conveniently located at Home -> Cells -> Format -> Organize Sheets -> Move Copy Worksheet. Whodda thunk it.