An Attempt to Answer a Quiz Question

Kyra heard the NPR Weekend Edition Sunday puzzle on the radio the other day: "Think of a common French word that everyone knows. Add a 'v' (as in 'violin') to the beginning and an 'e' at the end. The result will be the English-language equivalent of the French word. What is it?"

I thought, well, I don't know French, but I do know how to write a Python program that can look up the answer. Unfortunately, the puzzle appears to be a bit of a trick. I thought as much as I was writing the code. I was thinking, "I bet some people are going to be mad about the answer, because the question was worded in such a way as to be tricky."

At any rate, the code definitely does not result in the right answer, but it was a good exercise for me. First, I had to find two dictionaries robust enough to have a lot of words in English and French. Then I had to find dictionaries: I located the Debian operating system dictionary on my local computer, did a search in the package manager Synaptic for the French equivalent and any other English dictionaries that might help, and downloaded both. (The English alternative is labeled "insane" for its size, which "possibly contains invalid words (as well as words that are very uncommon)." [1])

Next I had to convert certain French characters into their English equivalent. To answer the question, I looked up all English words that start with "V" and end with "E", then removed those two letters, looking up the "word" in the modified French dictionary. It didn't work, as can be told from its output:

a
a
ah
aire
alu
an
ange
ares
as
er
erg
es
es
il
o
t

Yep, those are definitely not the answers. Some, when you add V and E back, aren't  even really English words. "Vte"? What is that? Oh, and I think A is listed twice due to the missing diacritics.

For anyone who's interested, I posted the code online. (It's not pretty posting a lot of code on Blogger, so here it is in a document.) Keep in mind I'm a new programmer, so I might not have done this task very elegantly. It is a short piece of code, though, and it could have worked.

One other note: If you attempt to run it, it may not work on your system. It's designed for Debian GNU/Linux, as it relies on dictionaries in a Debian system and it uses the "/" as the directory delimiter. Additionally, the "insane" dictionary makes it run a bit slow, and I did nothing in the code to help with that, like buffering the file, so you could run out of RAM or peak the CPU. It works on my system, which is all that mattered to me at the time.

Of course, a more robust program should be more careful and actually take user input in some fashion. Perhaps look up words that with other characters or use different languages. As it is, it's more of a "script" that does one thing and non-interactively. In any case, it doesn't even return the "right" answer!

[1] http://packages.debian.org/squeeze/wamerican-insane

Update in May 2012

Last year around this time I had decided to take a risk and leave my position at the University of Wisconsin Medical Foundation for a position at the UW School of Medicine, where I was hoping to be more involved with research. This endeavor hasn't turned out as I expected, so I've decided to move on. Starting later this month, I'll be working with CPM HealthGrades, a marketing firm that is offering more and more data services to their customers.

I accepted the position at CPM HealthGrades as a Data Scientist. This is a pretty cool opportunity in a relatively new field of data analysis, using some of the same technologies behind IBM's Watson computer, the computer that bested two top-rank Jeopardy winners. I'll be learning the programming languages Python and R, and I'm really excited to put these skills to commercial and research use.

As you may know, I attended the SAS Global Forum conference last month, and I plan to keep up with the SAS community as well, by writing papers, presenting, staying in touch with user groups, and maybe even writing a book. So I plan on keeping my SAS site up-to-date; however, I may not be developing much new. Perhaps I'll come out with a Python or R site to go along with it, with translations of code back and forth between the languages. Heck, maybe I'll write a SAS macro to write Python, or a Python module to write SAS. We'll see! Stay tuned for more details!

SAS and Keyboard Shortcuts

Base SAS has an easy way to add keyboard shortcuts that not only do X typical keyboard shortcut task, but also Y SAS program task (e.g., execute a macro). This can be a bit tricky to get to work right, but I've done it with CheckLog, PrintContents, OpenTable, and other interactive macros. However, I can't seem to figure out how to do some of these same things in Enterprise Guide.

Sure, it's easier in Enterprise Guide to open a table that's been run, but what about that library with 10,000 tables where I want to open only 1? It's like searching for a needle in a haystack, not to mention the load time just to see the tables in the library (very slow at 10k tables). It's so much easier to use OpenTable, and it's even easier to use OpenTable via keyboard shortcut. I have another keyboard command to open the properties of the table (a window that is quite severely limited in SAS EG, so perhaps that wouldn't be quite as useful anyway).

And the windowing commands seem to be gone, too. I can't edit a keyboard shortcut to jump from the program to the log: In Base SAS, I use CTRL + L for the log, CTRL + J for the program ("J" only because it's easy to reach and on the same keyboard line as L), and CTRL + K for executing the CheckLog macro ("K" for checK, I guess--again, it's on the same keyboard line as J and L). Just to note: I also have the CAPS key remapped to CTRL, so that's even easier to reach for my lazy efficient hands.

If anyone has experience with doing these things in Enterprise Guide, I'd love to hear feedback on how you get things like this to work - or if you just gave up and went with the new flow. My searches online have been fruitless up to this point, so I wanted to have this post available, at least for a point of contact in case someone else is wondering about the same things I am.

Another Odd SAS Error Message

I encountered the following error message last week:

"ERROR: Invalid value for width specified - width out of range"

The code that generated it was using SQL and the PUT function to convert an ID using a format, like so:

compress(put(id, idfmt.), '. ')

Where the IDFMT format originated as a user-created format, stored in a data set. The quick and dirty solution is to add the "?" modifier to the PUT function, which is oddly documented on the INPUT function but not the PUT function:

compress(put(id, ? idfmt.), '. ')

Now why wouldn't that be documented on the PUT function page?

And what about my original problem? I've tried varying the lengths of the input column, the format, removing the compress statement, and switching to TRANWRD or PRXCHANGE. I also tried to identify the records involved, but with 35.6 million records, that was far to slow. I know that the issue occurred in the second half of the data, but beyond that is too time consuming to look for the needle in the haystack.

Finally, I figured out a solution: The original format is based on an input data set, using the CNTLIN= option on PROC FORMAT to generate the format on the fly. The data set does not have the HLO (High/Low/Other) flag to indicate what non-matching (Other) starting values would be labeled as. I added it like so:

data custom_format;

    /* Output the original records */
    set custom_format end=end;
    output;

    /* Output an additional record */
    if end then do;
        start=.;
        label=.;
        fmtname='idfmt';
        hlo='O';
        output;
    end;
run;

Then the code using the format worked, and to boot it no longer needed the COMPRESS function to remove the blanks and numeric missing values (".").

Hopefully anyone else with a similar issue will find that one of the above solutions fits their need.

SAS Global Forum Papers Online

The SAS Global Forum papers are already posted online!

http://support.sas.com/resources/papers/proceedings12/index.html

Here are my papers:

Standardized Macro Programs for Macro Variable Manipulation

An Advanced, Multi-Featured Macro Program for Reviewing Logs

Feel free to read ahead if you're attending. It might help!