Archive for March, 2008

Massive Stock Datasets

Monday, March 31st, 2008

When data-mining, the first step is to obtain the data that you would like to mine. I have decided that I would like to try my hand at play­ing the stock mar­ket so it became nec­es­sary for me to obtain his­tor­i­cal stock mar­ket data. To that end, I have devised a method to obtain end of day results for every list­ing on NYSE, AMEX and NASDAQ since their incep­tion. The data is in the process of being assem­bled and I expect it to be com­plete within a few days. Current esti­mates expect the data to take up approx­i­mately 2GB, mak­ing it the largest sin­gle dataset that I have ever played with. Just hav­ing this much data makes my data hoard­ing senses tingle.

I’ll prob­a­bly spend a lit­tle bit of time putting the data into an easy to under­stand and use for­mat and then I’ll start look­ing for pat­terns. I’m hop­ing to throw my mod­el­ing back­ground and expe­ri­ence at the stock mar­ket to see if I can’t beat the sys­tem. If I can beat the stock mar­ket and make bajil­lions of dol­lars (or euro if the dol­lar col­lapses) that would be pretty sweet but if I don’t, at the very least, I expect to have fun play­ing with lots and lots of numbers.

As a sec­ond approach, since it turns out to be rather dif­fi­cult to get this sort of data in the first place, I’m half con­sid­er­ing the idea of clean­ing it up a bit and then reselling it myself.

Posted in computers, science & technology | 4 Comments »

Screen’s Clever Error Messages

Wednesday, March 26th, 2008

I am – and have for quite a while been – a huge pro­po­nent of GNU Screen for the many-fold improve­ments that it pro­vides to ter­mi­nal and SSH ses­sions. One of the things about screen is that it runs in two processes, one is a head­less process that redi­rects ter­mi­nal out­put of appli­ca­tions to it (server process) and the other is a process which con­nects to the server process allow­ing inter­ac­tion with and the view­ing of out­put from appli­ca­tions in the server process (client process). The advan­tage to this two process approach is that you can run screen from an SSH con­nec­tion, dis­con­nect the client process, leave and then later recon­nect a new client process to the same server process you started ear­lier, thus allow­ing ses­sion per­sis­tence. It’s a won­der­ful appli­ca­tion with many other fea­tures that make my life easier.

In addi­tion to being fan­tas­ti­cally use­ful, screen appears to have been writ­ten by some­one with a sense of humor as it seems to con­tain rather a few Easter Eggs. One par­tic­u­lar Easter Egg that I recently came across is that if the server process dies while the client process is still con­nected, you meet the error message:

Suddenly the Dungeon col­lapses!! — You die…

It’s a cryp­tic and omi­nous error mes­sage that might put you off a lit­tle if you don’t know what’s going on. A quick googling will show that some have acci­den­tally mis­in­ter­preted the mean­ing; thank­fully I could guess what it was and check google for confirmation.

Posted in computers | No Comments »

Numerology and Synchronicity in Restaurant Stubs

Wednesday, March 26th, 2008

When I see a num­ber or sequence, I have a ten­dency to decon­struct it into other num­bers or sequences; I do this with license plates, addresses, receipt stubs and all other man­ner of things. It is receipt stubs, specif­i­cally from restau­rants that I’d like to take as my dis­cus­sion nucleus today. Yesterday, for the first time, I got a steak and cheese from Theo’s Cheesesteak at the Rincon Center nearby and I hap­pened to be order num­ber 64. Upon see­ing 64, I inter­nally decon­structed it to 26 and spent the sub­se­quent few moments think­ing gen­er­ally about pow­ers of two. Today, hav­ing enjoyed yesterday’s cheeses­teak, I decided to get another and went, again, to Theo’s where I, once again, hap­pened to be order num­ber 64. Performing the same decon­struc­tion to 26 imme­di­ately reminded me that I had been given 64 yes­ter­day, allow­ing me to note that I had received the same num­ber from the same restau­rant, two days in a row. Coincidence: yes; syn­chronic­ity: I found it mean­ing­ful, so yes; evi­dence for numerol­ogy: oh come on, no.

This par­tic­u­lar syn­chronic­ity got me think­ing philo­soph­i­cally about the nature of sig­nif­i­cance and how we attach mean­ing to things. We humans are pat­tern find­ers, we do it very well and we tend to do it uncon­sciously. Patterns sim­plify the world and allow us to abstract things into eas­ier pieces. Since we use pat­terns to bet­ter under­stand tho world, we tend to do our best to fit things into pat­terns, even if it’s merely a coin­ci­den­tal pat­tern, which is fine because pat­terns do arise spon­ta­neously. People, in my expe­ri­ence, tend to con­flate the exis­tence of a pat­tern with the pres­ence of mean­ing. In the case of num­bers, I believe that the rel­a­tive ease of con­struct­ing arbi­trary pat­terns often leads peo­ple to attach mean­ing to things that are ran­dom in nature. I know that I’m guilty of attach­ing mean­ing where it isn’t due but, at least, I tend to be con­scious of and com­plicit in my misattributions.

The ques­tion now is whether or not I should get a cheeses­teak tomor­row, for pseu­do­science and all.

Posted in food & drink, musings | 1 Comment »

Wanted: Bandmates for Rock Band

Sunday, March 23rd, 2008

Yesterday, I wan­dered over to Best Buy and finally picked up a copy of Rock Band for my Xbox 360. As I have already estab­lish, Rock Band is awe­some, so I’m mighty psy­ched to have a copy at the place now. I wasn’t prop­erly set up for it ear­lier but now that we’ve got a pro­jec­tor and a proper sound sys­tem, we have an amaz­ing Rock Band setup. Rock Band, of course, is far bet­ter as a mul­ti­player expe­ri­ence than a one per­son game, so any­time that any of you folks want to stop by and jam with me, you’re more than wel­come to do so.

Posted in video games | 1 Comment »

Zipping about

Sunday, March 23rd, 2008

Since the recent demise of my poor Subaru, I have been effec­tively with­out a car. Being with­out a car isn’t such a big deal since I live in the city and can use my long­board for most of my trans­porta­tion needs but occa­sion­ally I have busi­ness out in Sunset, down the Peninsula or in the South Bay, which are not the most pub­lic trans­porta­tion acces­si­ble loca­tions. Up until now, I’ve been abid­ing lim­ited mobil­ity, incon­ve­nient mobil­ity and bor­row­ing my room­mates car but no longer. Friday, I signed up for an account with Zipcar. The whole process from apply­ing to accepted took about 20 min­utes and then, rather than wait­ing for it to come in the mail, I wan­dered to the local Zipcar office, a few blocks away, to pick up my Zipcard. All in all, it was about 30 min­utes between apply­ing and being able to bor­row cars. Yesterday, rather than bor­row­ing my room­mates car to run an errand, I picked up a Zipcar; it was a pain­less and easy process. Based on my cur­rent auto­mo­tive needs, Zipcar really does seem like the best approach and, based on my expe­ri­ences thus far, it seems like they’ve imple­mented their ser­vice correctly.

Posted in vehicles | No Comments »

Oh noes, Arthur C. Clarke is no more

Tuesday, March 18th, 2008

Arthur C. Clarke died today in Sri Lanka at the age of 90. I won’t even try to sum­ma­rize the amaz­ing things that Arthur C. Clarke has con­tributed to the world, I wouldn’t be able to do him jus­tice; if you care, ask wikipedia. Rest in peace Sir Arthur C. Clarke and thank you for all that you’ve given us.

I’ll leave you with Clarke’s three laws:

  1. When a dis­tin­guished but elderly sci­en­tist states that some­thing is pos­si­ble, he is almost cer­tainly right. When he states that some­thing is impos­si­ble, he is very prob­a­bly wrong.
  2. The only way of dis­cov­er­ing the lim­its of the pos­si­ble is to ven­ture a lit­tle way past them into the impossible.
  3. Any suf­fi­ciently advanced tech­nol­ogy is indis­tin­guish­able from magic.

Posted in books, science & technology | No Comments »

You’re not Irish this 17th

Friday, March 14th, 2008

Saint Patrick’s Day, 2008 is not on March 17th as usual but, rather, on March 15th. This shift is because March 17th falls dur­ing the Christian Holy Week. The Roman Catholic Church moves Saint Patrick’s Day when­ever it coin­cides with the Holy Week, a coin­ci­dence which last occurred in 1940 and will next occur in 2160.

In quickly polling around, it seems as though very few peo­ple are aware of the date shift. In fact, even my wall cal­en­dar has the date wrong. Don’t get me wrong, I like a good drink­ing hol­i­day as much as the next per­son but if you’re going to hijack a hol­i­day and use it for drink­ing, you really should do the hijack­ing correctly.

Drinking on the wrong day makes the as-yet-unressurrected baby Jesus cry. Don’t do it!

Posted in art & culture | 3 Comments »

Fuck Daylight Savings Time

Sunday, March 9th, 2008

In all seri­ous­ness, I don’t know why we need day­light sav­ings time. All it does is make me have to change my clocks twice a year and likely miss at least one sched­uled event shortly after the tran­si­tion. Sleep sched­ules don’t have much to do with day­light so why are we try­ing to force to two to coin­cide. We really need to get rid of these pathetic ves­tiges of days past and make our time get with the time. I hereby, pub­licly request that Daylight Savings Time ten­der its res­ig­na­tion, effec­tive immediately.

Posted in politics, waksman | No Comments »