Send Your R-Trees Packing

So long ago now I was looking at R-Trees and how scary the ones created by GiST indexing in PostGIS looked. I had actually already written a packing algorithm at the time - which I called Median-Split packing, which made some pretty nice looking indexes. Here is a picture of the index of the same data I was using before (the roads of British Columbia) when packed using Median-Split:


And here ! is a zoomed-in look at the densest area, the Greater Vancouver Area - approximately one third of all the road segments in the province are within this area which is on the order of 1/100th of the area of the province.


I have yet to do empirical tests to determine if the packed index works any better, but from a purely aesthetic point of view I'm happy with my packing algorithm. It works in O( n*log(n) ) time, using an algorithm similar to qui! ck-sort. I am confident there will be a measurable improvement! in perf ormance in terms of the number of page-accesses required for the packed index vs. the regularly constructed index, however the real question is, once caching and what-not is factored in, can you see a significant difference in query time in the database.

Lots of other people have thought about packing R-Tree indexes before, STR-Pack and Hilbert-Sort do perform similar operations in similar time, but I feel strongly that Median-Split does a better job than they do, particularly for region queries, which are common for map rendering (as opposed to point queries). I don't yet have any proof of this but it just "feels good". To my knowledge no-one else has published an algorithm that works the way Median-Split does - I may look into publishing it.

Shoehorning an index packing mechanism into PostgreSQL is a daunting task, so before I ! go down that road I'm going to do some simulations to determine if I'm right about the performance in terms of page-accesses.

median split

March, my birthday, and I'm still here!

Hi Readers! I am still here responding to people's requests for homework help, I just don't post the answers on my blog as much anymore. But rest assured, I am still here answering your emails.

Also, to celebrate my birthday this week, I invite you to partake in this fun online gaming experience called "Auditorium"!

http://www.playauditorium.com/

I enjoyed it so much that I played it all the way through on my first sitting!

math help for free

The turning of the histogram

I have been off for some time now, and I thought that it would be high time to post something on gnuplot. I have chosen an easy subject, but one that could turn out to be useful. At least, I have seen people searching for a solution to this problem with the histograms. If you needed it in the past, you have probably realised that gnuplot can make only vertical histograms. But sometimes, this is not the best way to represent data, and a horizontal version would be much better. For one thing, the horizontal one might take up much less space. If you want to learn how to produce the figure below, keep reading!

Making the graphs will require some work, but everything is quite straightforward. What we should note, however, is that we will still produce an upright image that we have to rotate later. If you set the terminal to postscript, you will probably use the figure in LaTeX, where you simply have to tell the compiler to rotate the figure by 90 degrees. If you set the terminal to a raster format, or print the screen, you can rotate the image in many applications, but if you want to adhere to the command line, you can, at least, under Linux, use the convert command as

convert -rotate 90 figure_in.png figure_out.png

The figure above was p! roduced based on the following data file

1989 0.1
1990 0.2
1991 0.2
1992 0.05
1993 0.15
1994 0.3
1995 0.1

(We have seen these data many times, you should know it by heart by now!) After this interlude, let us see the code! I will discuss it afterwards.

reset
set key at graph 0.24, 0.85 horizontal samplen 0.1
set style data histogram
set style histogram cluster gap 1
set style fill solid border -1
set boxwidth 0.8
set xtic rotate by 90 scale 0
unset ytics
set y2tics rotate b!  y 90
set yrange [0:0.35]; set xrange [-0.5:6.5]
set y2label 'Output' offset -2.5
set xlabel ' '
set size 0.6, 1
set label 1 'Year' at graph 0.5, -0.1 centre rotate by 180
set label 2 'Nowhere' at graph 0.09, 0.85 left rotate by 90
set label 3 'Everywhere' at graph 0.2, 0.85 left rotate by 90
p 'pie.dat' u 2 title ' ', '' u ($2/2.0+rand(0)/10.0) title ' ', '' u 0:(0):xticlabel(1) w l title ''

The first line after the reset is required, because we have to make our key ourselves. We simply specify the coordinates and that that we want to have a horizontal key (i.e., one in which the keys are placed to the right of the previous one), with a length of 0.1. If you want smaller or larger space between the keys, you can modify this, by adding the spacing flag to the set command. You can see the exact syntax by issuing ?key in the gnuplot prompt.
In the next four lines, we set up our histogram. The content of these lines ! might depend on what exactly we intend to plot. For more on th! is, chec k out the gnuplot demo page! We then rotate the xtics by 90 degrees. We do this, because the whole image will be rotated, and by rotating the xtics, we make sure that those will be horizontal at the end. For the very same reason, we also unset the ytics, and set the y2tics. We also set the y2label and xlabel. This latter one is empty, but we still need the space for it, so we set it to ' '. Beware the white space!

The next important step is the setting of the aspect ratio of our figure, which will be 0.6:1. Having done that, we introduce 3 labels: one for the xlabel, and two for the key. The placement of these labels is somewhat arbitrary, and depends on the particular terminal that you use. But only these three numbers and the coordinates of the key need any tweaking, really. At the very end, we plot our data. I plotted the second column twice, with an added random numbers, so that we can have ! two columns at each data point. Note that we plot the first column, too, but pass it to the xlabel command. By doing that, we can automate the labelling of the xtics, using the first column in our data file. Also note the specification of the titles: in the first two cases, the single quotes include a white space, while in the third one, there is nothing.

histograms examples

Rocky Run

Rocky Run was one of my favorite places. It was a place to have a beer with friends, a place to chat, and a place to eat a simple (but satisfying) meal. I am certainly not the first to blog about this (Columbia Talk, Howard County Maryland Blog). I first walked into Rock Run in 1996, and I kept coming back. Here are a few reasons why:

Quirky:

When you walked into Rocky Run, you knew this was not a national chain. It never took itself too seriously, and this was reflected in the dé! cor. One of the few restaurants in Columbia that had its own brewery, there was a quirkyness about the place. The enormous hot sauce collection, the Elvis booth, the Beatles room, the Buffett room. The peanuts on the floor in the bar (great when fashionable, even better that they stuck with it when it fell out of fashion). The foreign language tapes on continuous loop in the bathrooms. The Rocky Cocktails (I confess, I had a Jamaican Bobsled just last week!) The endless trivia games.

The Food

Since it opened, Rocky Run’s menu has always been full of offerings. Standouts in my mind where the chicken wings, which always seemed larger than the wings served at other restaurants in town. Crab Pretzels on Fridays during lent. The salads (the cobb salad was a personal favorite), of course the burgers, and the impossibly large “loaded” baked potatoes. The Cuban pannini was a late, but welcomed ! arrival on the menu.

If people asked me about the f! ood at R ocky Run, I would always respond “It’s the best comfort food in Columbia.” I think that said it all.

The Staff

Over the years, I have gotten to know many of the people behind the bar and waiting tables. It all started with Alex, who on that first day I walked into the bar, shook my hand, introduced himself and asked me what I would like to drink. I quickly came to know him and many others. The always bubbly Liz, Adam, who insisted on being called Fish. Mary Ellen, who was in training to become an EMT before she trashed her knee. The always nice Amy. Ruthie, who had issues, still managed to have a good time and smile.

During the late 1990’s Heidi and Sean tended bar on Friday nights, and they always managed to serve an impossibly large crowd. At that time, Margarita Maggies was still open, but not doing well, and people would park in the Maggies lot to go to Rocky Run. He! idi and Sean performed their craft well. They were fast, charming, and seemed to enjoy themselves.

As time passed, some interesting bartenders made their way through. Art was always fun with his libertarian views and his dislike of public displays of affection. Jeff was a master of many subjects. Henri, was just an all-around good guy; smart, witty, and a die-hard Wizards/Bullets fan.

There were a few that found love at Rocky Run. Caleb and Christy are still together and Tommy and Jenny were an item.

But, over the years, three stand out. Jason, who started as a busboy, became a waiter, host, bartender, bar manager, store manager, and I’m sure is still doing well. Jason, I have seen you grow so much over the last twelve years. I am honored to call you friend, and wish you well. Christy (as mentioned above), you have been part of my nights out for so long that I will miss you. Dave, you are a great man and I am sure good things! will happen for you.

I know that I have missed so ! many (oh , I remember another, Wendy!) who wore the bright orange “Rocky Rookie” shirts on their first day, I will miss you too, as well as everyone in the kitchen.

The Smartest Patrons Ever Known

I am serious about this point. Since the day it opened, Rocky Run always ran the NTN/Buzztime trivia games; Half hour trivia contests that linked thousands of restaurants across the United States. Rocky Run attracted a clientele that was adept at these games (and for a long time, I was one of them) and often times Rocky Run would be ranked in the top 20 in the nation. Occasionally, a person would arrive at Rocky Run just based on seeing the restaurant on the trivia boards at other restaurants. During the late 1990’s, Rocky Run and Nottingham’s were engaged in a battle of trivia supremacy. Trivia players were courted. Scores were displayed; cell phone calls were exchanged after games.

What ! gave this rivalry depth was that at Nottingham’s, it was known that the trivia players typically shared answers. At Rocky Run, the unwritten rule was that you played your game. Shouting out answers was considered bad form. That is not to say that the trivia players were stoic and silent. After playing for a while, players got to know each other. Most were witty, some were outright funny, and all liked passing time together.

So Moron, Phlegm, Farkle, Redgirl, Karl, and all the many people who got the trivia bug, I will find a place to play, and I hope to see you on the board too. I will miss you all.

Beyond the trivia, the folks at the bar were generally well versed in the topics of the day. You could always have a good conversation about national or local news. Just smart, wonderful people.

Where do we go from here?

I was at Rocky Run just last Friday. A friend of mine had just! accepted an offer for a new job, and we went out to celebrate! . We ha d dinner, discussed the job, what’s new in our families (my son wears a Size 1 shoe, and he starts kindergarten this month), sports, and the local politics scene. We had arrived at 6 o’clock, and the bar was busy. When we left at 9:30, the place was nearly empty. Christy, Dave and Jason were there. We said hello and things seemed all right.

I guess all we have are memories now. I can still see Joe and Ellis sitting at the end of the bar. Bobby holding court at one of the short tables. Don and Deanna discussing their latest travel plans. Bill and the trivia faithful playing trivia and chatting up each other. Jason and Terry stopping by for dinner. Some stranger saying out loud “Oh, I LOVE this song,” and me saying to myself “I love this song too.”

Yesterday I stopped by to take the picture to accompany this post. The owner came out and said hello. We talked for a short while and then I just had to ask: What! happened? He told me that business had really dropped off drastically in the last four months. I thanked him and inquired how the staff were doing. He had told me that more than a few had already found jobs. I was relieved to hear that. I just smiled and thanked him again. He smiled back and thanked me, then quietly went inside. I took a minute and sat on the bench outside. As I sat there, four cars pulled up, and each asked the same question: What happened? I told each what I had heard, and they all, hesitantly, walked back to their cars, muttering “This is awful…This was my favorite place…”

I don’t know where or when my friends and I will get together for happy hour, but when we do, we will loudly toast the memory of a place that enriched our lives (better said, provided an atmosphere that allowed us to enrich each others lives).

I hope we can find a place that will have that same feel; that will give us a Seco! nd Chance.

course compass answers

hurry up and get here

is it Spring yet? this Winter is killing me. i'm not even in a wintery place and i'm ready for it to be over. i need some renewal, some rebirth. i feel like Springsteen....i wanna change my clothes, my hair, my face. a strange thing to feel restless whilst wandering. maybe i'm just home-sick. but when i get home, i want to purge. i need to shed the old stuff and exfoliate my life a bit. it's time to let the new stuff really OUT.

a change is coming. or will come, after i start working smarter towards it. a big personal one for me, actually. i'm making a break for it! . i've been wandering through the soundscape of the music world for several years now....floating along, making recordings, trying to improve my skills....but with no real plan. and it's time to step up with a plan.

the past few weeks i've been slowly coming to realizations that others have been easy to see. but i was inside and couldn't see it. simple things that i could say and do to help improve my status as an indie. spreading the word, the new music, everything.

but i say "i'm no business person. i don't know how to do this!" but that's a ridiculous statement. it doesn't matter that i didn't know how to do any of the stuff i now know how to do, before i learned how to do it. i simply did it. and now i know how to do stuff. and this time is no different. it's just time to be the person with the vision. and to write this vision down in words on a piece of paper. and to truly develop a real, long-term plan with definite goals, rather than just wander! ing along, hoping to be seen by a person with more money, powe! r, and c onnections than myself.

and finally, a distinction. Kristin Putchinski can no longer be on stage with ellen cherry. it sounds weird, but ellen cherry has to start standing on her own.

congreve cube

Flow Chemistry to help the Pharmaceutical Industry

We have recently started a new research project in collaboration with Mimi Hii from the Department of Chemistry (Enabling Oxidation Reactions on a Large Scale: Combining Electrochemistry with Flow). The project involves partners from Pfizer Global R & D.

In a nutshell, the project is about creating in situ reactants (oxidants and reductants) using electrochemical methods to react with pharmaceutical intermediates in a flow system (that is continuous production rather than batch operation). We hope that this will lead to much improved selectivities and yields with much reduced waste formation for many important chem! ical transformations.

We are currently filing a patent application on our ideas with Imperial Innovations.

chemistry help

Process Goals and Fun with Graphs!

It is timely that I came across this nice post today about goals, because I had been thinking yesterday about listing out some goals on my wall that are not topic-specific, so that I can refer to them throughout the year as I work with my kids on different assignments intended to address one of those life (or processing) skills.

Examples for Geometry:

• Visualization of parts vs. whole (Isolating info from one part of a diagram; combining info across multiple layers / perspectives.)


• Mechanical precision (Measuring and constructing lengths within 1 mm error and angles within 1 degree error; building solid models that won't fall apart.)


• Comprehension of diagrammed instructions (ie. for building anything)


• Attention to details in written instructions


• Judgment of reasonableness (of any physical or vi! sual quantity)


• Effective written and oral communication


• Perseverence and resourcefulness


• Team work

Examples for Precalculus:

• Interpretation of data (Understanding trends / real-world significance / impact.)


• Fluidity with technology (Using graphing calcs fluently and flexibly.)


• Flexibility / creativity of problem-solving approaches


• Risk-taking and reasoning about the unfamiliar


• Precision and clear step-by-step organization of thought process / math work


• Increased self-management of progress, frustration level


• Effective written and oral communication


• Perseverence and resourcefulness


• Team work

Maybe it's just me being cheesy, but I think both the students and I might benefit from my posting these goals on the wall -- especially if I do actually highlight different skills throughout t! he year, as is appropriate to a given lesson, so that the kids! would f eel like they're not just learning content for content's sake and we are working towards a bigger picture.

My lists of goals are pretty generic (not really even math-specific, for the most part... Most are just good habits of mind...), but they are reminders of what is most important to me. Naturally, between the freshmen (Geometry) and the juniors (Precalculus), there is quite a bit of a "life experiences" gap. So, my goals for the two groups are pretty different as well. My juniors don't need as much for me to hold their hands on following directions, but they might need a nudge to be more creative and/or persevering. Versus the freshmen, whose first task of this year is to learn to consistently read and follow instructions. :) They each are at a different developmental stage with their meta-learning.

We have an opportunity each day to impact kids, mathemagically or otherwise.

-----------------

By the way, I found some nice gra! phs for my graph-reading lesson that is coming up. They're neat, even though they are not terribly complex in a mathematical sense. (The textbook's got a couple of nice complex graphs to use for rigorous classwork exercises, so I feel like I could afford to spend the rest of the period looking at interesting -- albeit simpler -- graphs with the kids.) They have to do with the mobile market, digital textbooks, and digital music. I plan to have fairly open-ended discussions about trends in those graphs, who might be interested in reading them, and what their implications might be for those people. And then maybe end with why these kids should finish college.* It could end up being a total flop of a lesson hook, but I am curious to see what these juniors can bring to! the table.

If I have time, I am also thinking abou! t adopti ng a middle-school post-it bar graph activity to composite bar graphs and scatter plots. I would give every boy a yellow post-it and every girl a pink post-it, and we would start a bar graph template on the board -- for example, models or brands of cell phones. They would go put their post-its on the board, in the appropriate column, and we would re-arrange the post-its to stack the pink post-its on top of the yellow post-its, in order to create a composite bar graph showing how many boys, girls, and total # of students have each type of phone. You can also do this exercise with scatterplots (for example, height vs. foot length) and see at a glance 1. what the overall trend is within the class, and 2. what the gender-specific trend is within the class. Using only colored post-its! And requiring maybe only about 10 minutes, discussion included. ...I'll try to squeeze it into my Precalc lessons this week and tell you lurkers how it goes.

*From what I hear, many Salv! adorean private-school kids end up dropping out of college in the States, because they either lack the academic skills or -- more commonly -- can't deal with the fact that they no longer have a live-in maid and a designated chauffeur. This waste of an opportunity is terribly sad, considering that their parents can afford them to obtain a U.S. college education, while the majority of people in this country are living in poverty and many are starving. :(

But, can you really blame the kids for this injustice?? They are a product of the system (of huge disparity in wealth). sigh.

bar graph template for kids