Josh Highland’s Blog

Its not secret, I love social networking, I cant get enough of it. I also love programming and anything internet related.

I’m not sure how I came across it, but a PHP based, Object Oriented RSS caching tool named SimplePie caught my attention.

Previously I had been using a tool called Last RSS, but I found that Last RSS could not handle ATOM feeds.

Digging into how SimplePie works, I found that I could merge several feeds together, sorted by post time, my first thought was to use this to aggregate and cache all the different music related news feeds that and going to add content to notpopular.com v2.5 (when ever I finish it). I started thinking about it a bit more and thought it would be cool to use SimplePie to mash together all of the different RSS feeds from the different social networks I am on.

It wasn’t that hard to use SimplePie and the various social networks to make something really cool.


//establish the feeds
$pownce = new SimplePie("$pownceRSS_url");
$flickr = new SimplePie("$flickrRSS_url");
$twitter = new SimplePie("$twitterRSS_url");
$digg = new SimplePie("$diggRSS_url");
$youtube = new SimplePie("$youtubeRSS_url");


//merge them all together!
$merged = SimplePie::merge_items(array($pownce, $flickr, $twitter, $digg, $youtube));


Then all you have to do is look over each item in $merged, and output the appropriate info you want.

You can see what I built over at www.JoshHighland.com

One thing I really found great was the API that was on the SimplePie site.

After I put this all together, I found out that there is a term for what I had just built. People are calling them “LifeStreams“, which is a very appropriate term. I thought I had invented something, but guess LifeStreaming is like fire or a spear, given enough time people all over the place will discover it on their own.

I encourage you do set up a LifeStream for yourself and post the URL in the comments below. It’s fun!

My LifeStream: www.JoshHighland.com

Why do I have to magically fit all of the geek stereo types? So what if I like MySpace more then any other social network, I adore my iPhone, and I own a social network!

Something must have worked though, because I was able to land an awesome wife!

At work, I have been lucky enough to work directly with Luis Majano, the author of an awesome Object Oriented Framework for ColdFusuion, ColdBox (I have blogged about it before)

Using the Coldbox Frame work really took my CF coding to the next level.

At home, I am not lucky enough to have access to a Coldfusion server. I run everything off of a dedicated PHP machine that I have had for some time now. Mostly due to the cost of the CF licenses, and the investments I have already made in the running so many sites in PHP.

After using ColdBox, I wanted to find something similar that would really help me write better code, and do it more quickly. I first looked at cake and wasn’t that excited about it, it seemed more complicated then it should be.

After doing some more research, I came across Code Igniter. Code Igniter has turned my PHP world upside down. Its easy to use, straight forward, and has a lot of build in functionality that would take me forever to achieve on my own.

Working in an object oriented frame work just makes me feel like a more effective programmer for some reason. Maybe its because I feel more organized and can really see how fast I can put applications together using it.

The user guide, video tutorials, wiki, and forums for code igniter all rock. There is a wealth of information in there that I can draw off of.

I am re-working notpopular.com 2.0 once again (at this point maybe i should call it notpop 2.5), and I am using Code Igniter to do it. Lets hope that my newly peaked interest in object oriented PHP development will carry me though the development and launch of notpop 2.0 (2.5?!)

Recently I have been looking for a solution to allow users of notpopular.com to have photo albums of unlimited size. Its a popular trend in social networks, and its way cooler then the 5 tiny images that are allowed on notPopular.com (1.0)

There are many technical problems that associated with having this kind of open system. One of the biggest problems is storage (disk space), and load on the web server. Each image takes up space on the hard drives, and requires attention from the web server when an image is requested, taking attention and CPU from other more important processes.

After looking at several options, I came across Amazon S3 (Simple Storage Service). Amazon S3 is an online storage web service offered by Amazon Web Services. It provides unlimited storage through a simple web services interface. Data can be easily stored and retrieved at any time, from anywhere on the web. Amazon charges in proportion to the amount of data stored and applies charges for sending and receiving data. A perfect solution for what I’m looking to do!

S3 would store the data, and would handle the load of the requests coming from users, allowing my web server to focus on more important tasks like PHP processing and MySQL queries.

Here is the price break down for amazon S3 usage:

• Storage
• $0.15 per GB-Month of storage used
• Data Transfer
• $0.10 per GB - all data transfer in
• $0.18 per GB - first 10 TB / month data transfer out
• $0.16 per GB - next 40 TB / month data transfer out
• $0.13 per GB - data transfer out / month over 50 TB
• $0.01 per 1,000 PUT or LIST requests
• $0.01 per 10,000 GET and all other requests

Using the calculator tool they have, and looking at the notpop data and projected traffic, my monthly costs would be around $20 a month, and even if things doubled in time, it would still only be $40 a month. Very reasonable for the speed and reliability that a company like Amazon offers.

I found some great php classes to work with S3, particilarly one from Geoffrey P. Gaudreault over at neurofuzzy.net

I am also planing on using some CNAME records on the notPop server to mask the amazon URL. Should be easy to do.

Last night I started to sew Amazon S3 into the pending notPop 2.0 code base. I will make some follow up posts about how it goes once I really start using it.

I realized last night that I need to stop being a lazy web developer, and get off my ass and make something happen.

I’ve sat back and watched all of these awesome website applications come online, like digg.com, pownce.com, facebook.com and more, and I say “thats awesome, I wish I would have come up with that”.

The truth of the matter is I CAN do something like that, and I have the potential right at my finger tips for it to be big. For the last, almost 6 years I have developed a small but fun social network, notPopular.com, with over 25,000 members.

I’ve been admittedly lazy with the site and have let it go stagnant over the last few years. Its dated and worn. A number of time I have started building the 2.0 version of the site, just to trash it and start again. Last night I fixed a javascript bug that I had known about for 2 years… talk about lazy.

I work in burst of development, an idea will hit me and I will code like mad for a week, and then not touch it for 2 months. This is laziness. I need to stop this. I need to be steady with my development.

I love to code, I love new ideas for sites. Social networks are very interesting to me.

Many times I have felt left out of the online cool club, and I would think to myself, “If I lived in the bay area, things would be different, I could collaborate with people and make the next digg, pownce, or twitter”. or “I program all day for ‘the man’, I don’t feel like coding when I get home”. I’ve realized that those are just an excuses for my laziness. I need to get off my ass and do something about it. I need to develop web applications, if thats what I want to really do with my life.

This is my promise to myself, to complete the web projects I start, and to become consistent with their development.

If someone is reading this and has some awesome ideas for notPopular.com, speak now…. I’m on a roll

I have been really happy with this new blog software. So happy that I want to give you your very own blog account on the notpop network.

If you’re interested, please send me an email to joshhighland {at} gmail {dot} com and we can make it happen for you.

You blog account would be in the form of notpopular.com/blogs/YOUR_NAME_HERE

Its free, what do you have to lose?

I have gone away from my home made blogging software and embraced the multiuser version of wordpress. I went with the multi user version so I could offer blogs to the members of notpopular.com. My wife Amy is the first one to help me test it out. Please visit her blog and tell me what you think of the new blog system.

I have been talking about notPop 2.0 for a long time now. some people think its just a myth.

The truth is I have been working on notPop 2.0 for some time now, and its going to blow notPop 1.0 out of the water.
Why is it taking me forever? Because every thing from the ground up is new, and massive. The time to code and test each piece takes a while.
notPop 1.0 was developed when I was in college. I had a lot of free time then. These days, I work a full time job coding all day. At night I split my time between remodeling 2 houses, maintaining 5 websites, and spending time with my wife. Im lucky if I can get 30 minutes or an hour a night to work on the new notPop.
I have actually been thinking about taking vacation time off of work devote more attention to coding the new site.

So to the skeptics, Im working on it. Its just going to take a bit longer, so please hold in there and send me any suggestions that you might have.

This weekend I had some major problems with my web server. Here is a brief log of events.

Saturday 07/16/2006

• 3:00pm: I try to go to one of my sites, "can not connect to database". It seems that all of the connections are in use. Weird. I check the server… 500 open db connections? I reboot the mysql server, things seem like they are working.
• 4pm - 5pm: I go out to eat with my friends. Using my phone, and a ssh client, I check the database server, things seem to be ok. (damn im a nerd! im checking my linux database sever via ssh on a cell phone!)
• 6pm - 7pm: mysql server still running. I look at the message board on notopular.com, "database error". After doing some research, I realize that the harddrive that the databases sit on is 100% full. Not good. Looking at the files, there are a ton of error logs that have been generated. I clear up some room on the drive, reboot the sql server. Sweet now I can write to the DB again.
• 8pm - 10pm: Looking at my message board, "Cannot connect to DB", some more research, I realize that some of the tables for the forums application are corrupt. great. I break out the good ol' "myisamchk -r" command and get to work. A bunch of the indexes are totally hosed. After everything is done running, I can connect and view the database, I back everything up.
• 12 midnight - 4am: I check the message boards again. I can view everything, but I cant post to the forums! the table that contains the message text (a HUGE table, 670,000+ records) has something wrong with it, I cant write to it. Again i break out "myisamchk -r" but i get an error. "Please use the -o flag, -r can not repair this table". CRAP! I need this table! its the guts of the message boards!. I know that the -o flag is going to take a while, so I bring down the apache webserver to gain some CPU power, and then run "myisamchk -o" on the table. I figure out how long its going to take… so I reniced the process to turbo charge it, and cut down on the time… by cutting down on the time, I mean its only going to take 3 hours to run!
• 4am - 5am: The rebuild of the table indexes has worked! All of the records were salvaged! I bring the webserver back up and start checking. Everything is looking good. Back up the database locally and offsite.
• 5:30am: the sun is coming up. im going to bed.

All in the night of a developer/webmaster/systems administrator

I got an advanced copy of the new AFI album. major bum out. Read my review here.

http://www.notpopular.com/review.php?review_num=170