Saturday, September 06, 2008

And with one final click, the project is complete.
I've uploaded the source of my project to Google Code.
This whole project has been quite the experience.

I plan on continuing this blog, don't expect timely updates though.
I'm doing Grad work full-time at WPI (www.wpi.edu) as well as some part time work.

Sunday, August 10, 2008

IT WORKS!!!

And not a moment too soon.
I've finished my day job for the summer and have been pouring over the code the last few days trying to isolate the last two bugs.
I've also received word that I don't have 8 days left, I have 1...so I'm a bit rushed but still hopeful.

As part of my bug fixing for the hit code (which works beautifully now) I was able to remove all of the unnecessary mutex locking from within the cache. This is a nice speed boost as there is no longer any need to lock and unlock when working with the cache. It just works (tm).

I have a fairly rough implementation of the invalidate code that looks something like this:

if(anything changed) {
nuke_the_cache();
}

It works, and it allows for multiple instances to use the same cache.
My next task is to make this a bit more refined (per table invalidation) by using 'namespacing' within memcache. The idea works like this:
For each table, store a "0" in memcache with 'db_name:table_name' as the key.
Each time a table changes, increment that key.
When formulating the key for a query result, include each relevant get('db_name:table_name') in the key hash.

The nice thing about this approach is no matter how much data is in the cache,
invalidating is always O(n) where n is the number of tables involved.
The previous query_cache could slow down a MySQL instance in a write-heavy environment due to searching for and invalidating entries in the cache when a table changed.
With this implementation, everything is non-blocking and when a table changes a quick call to 'memcached_increment()' is all that's needed. Memcached will remove stale entries after they've expired, since once the namespace has changed they will never be read again.

It looks like I will not have time to refactor this into a plug-in during the summer of code timeline. But I've met and exceeded most all of my other goals so I'm not too worried.
I'll have a tarball, a diff, and hopefully a checkin with bzr done tomorrow to prepare for my final review.

Best of luck to all!
We're almost done!

Monday, July 07, 2008

IT WORKS!!!

I've been stressing for the first day of midterm review and accomplished quite a bit.
I've managed to work out some of the bugs to the point that it now caches!
It doesn't hit yet but I might just be able to pull that off by the end of the week.

I've hit my milestone and I'm much more confident about the review this week.
I haven't heard back on my pre-review review from my mentor, but with July 4th holiday weekend I really can't blame him, I celebrated too.

Went to a local amusement park and visited with my friend Eric who I haven't seen much of.
Definitely need to brush up on my air-hockey skills after this project is completed.

Just eight more weeks to completion!

Thursday, July 03, 2008

I finally got the Eclipse debugger to play nicely with MySQL!!!

Steps:
make clean
./configure --with-debug (and I modified my flags with -g as well, tho it's not required)

then under Eclipse I did a 'make all'
used --gdb --one-thread when running
be sure to close the 'registers' pane or else MySQL will crash and require a kill -9

I can now step through my code and see it in action,
it really is doing just what I thought it should!
(tho I found the MySQL GUI tools do a lot of things I didn't know about)

I'll be submitting an early review to my mentor tonight.
I'm not too familiar with submitting a diff so I'll probably attach the full source files as well in case I mess it up.
As of tonight I have surpassed 100 hours working on this project!
In a way I wish all this really was for JUST a t-shirt, adding money to the mix is both an incentive and a curse. I can always continue working on the project, even if I don't make the cut though... :)
I just hope my results add up to the effort I've put in.

Monday, June 30, 2008

As midterm reviews approach I've been working on cleaning up any and all messy hacks so I can have a nice presentation. Last week I started using the Makefile.am instead of bash scripts. This week I tried cleaning it up further so that libmemcached is compiled automatically as needed instead of as a pre-installed dependency. Now any developer can download and compile my modifications without having to jump through hoops.

I'm beginning to feel as though the build tools are more difficult to edit than the code itself.
After numerous 'unable to find header' errors I think I have finally fixed every INCLUDE declaration necessary. I think this has put me a bit behind for the midterm review and caused a bit of stress, but I think this will save me a great deal of time overall.

Had my first segfault while working on the project last week, it was a pain to track down without a debugger. The Eclipse debugger doesn't play nicely with MySQL on my machine at the moment. I've been poking around with Netbeans as well though I haven't tried debugging with it yet.

I am still shooting for being able to cache and hit by next week.
That is my goal for a "Hello World" of sorts.

The one question that worries me on the review sheet is along the lines of "have they earned $2,000?". I have a great deal of difficulty associating time and code with money. I've been coding since middle school... I code in my spare time, I code for fun, I code at work, but coding and money aren't linear in my mind. If you based it just on hours spent, I don't think I've put in 200 hours into this project yet, though I am getting close.

I worry too much, back to coding!

Tuesday, June 24, 2008

Alright! Thanks to my mentor Brian I now have a semi-clean build procedure in place.
I just type 'make' and it compiles both libmemcached and mysql together nicely.

The previous setup was rather hackish and involved some bash scripts I'm not proud of, but it worked. I suppose since Google sent me a book on how to write Beautiful Code, I should really be writing code accordingly.

Just two weeks until 'midterm review' ... *panics*
I feel like I am making good progress despite my lack of time.
I'd really like to have something functional in the next two weeks, bugs are inevitable.
My mentor said that it was important to find my "Hello World!" as he put it when working on this project. I'm not sure how to define that though, it compiles, it runs, it connects to memcached, but is that Hello World worthy enough or am I worrying too much?

I'm going to stick to my goal of trying to have it at least try to cache and hit before the review, but I will likely fall short.

I should blog more, the weekly updates are nice, but a more frequent update cycle would help to keep me on track.

Happy coding!

Monday, June 16, 2008

I'm finally starting to feel at home with my build environment.
I have my timeline, I have my tools, and I have motivation.
I've gotten Eclipse/CDT working just right, I'm making small iterative improvements, compiling, debugging, and then repeating.

It doesn't quite feel like a Monday, I worked from home today, and had a nice long talk with a friend of mine this evening. Everything seems to at ease for a Monday. I only hope they'll forgive me for turning in my report a few hours late ... again.

The code:
-Compiles!
-Runs!
-Doesn't cache or hit.

But there is definite progress, the framework is all laid out, now I just need to fill in the methods and tie in memcache. I should have working software ready for midterm review, and even better software for the final review. It would be nice to see this code in production someday.