While setting up this site, everything was working great until I started to test the server under high stress using “Siege”. After running the test with 50 requests/second, the server started to buckle and after 15 seconds, Apache became unavailable. I spent about 2 weeks trying to figure out what could cause this problem, but until today, I have been unsuccessful. I will first explain what I tried and how each change I made, either did nothing or caused different problems making them not a viable solution.

My first thought was to optimize Apache to handle more connections at one time. Apache’s HTTP Server website has a Performance Tuning section. After making these changes and restarting Apache, the Stress Test still caused the server to die.

Next, I thought it might have been the mysql server that was the issue. Even though, the mysql server wasn’t getting high load, I decided to fine-tune mysql as well. Since we have a dedicated mysql server, I used the configuration that was located in mysql-huge.cnf. This configuration is designed for servers that are just used for mysql. These changes seemed to help, but it wasn’t even close to what the site should be performing at.

I then thought it might be the amount of RAM on the server. I resized the cloud server to 4G of memory, but there was still no change. I resized it back down to 1G of memory which is what we thought it should be at.

After looking into more server configurations as the problem and getting nowhere, I decided that I would see if Joomla was causing the issues. You should know that this is all being done on a development server. I started by removing the Javascript files that were being included to display the client’s Ads. They have a total of 32 JavaScript calls and takes 10 seconds to just load the Ads. Removing them helped speed up the site, but the high load server failures were still happening and I added their Ads back in.

Now, the server is using Varnish for caching the content. It should be holding onto a static version of the site for 2 minutes (what it defaults to), but it wasn’t keeping the cache. Apache was still getting called every time you load the site. Varnish was still pulling the page through cache, but it was not keeping the cache. I have Varnish running on all of our servers including my personal server farm, however, I couldn’t figure out why this site was not keeping cache.

I tried removing some of the third-party modules that could have been incorrectly programmed and causing serious performance issues. After removing all of the modules on the homepage, the problems still existed. I got to the point where I didn’t think that it was anything to do with the server or how the site was pulling its content through Joomla, but I felt that there was something on the Joomla Core that was the problem.

I spent the next few hours researching what Joomla could be doing that was making the server die. After finding this blog article, I had an idea. His problem wasn’t my problem, but something that he wrote got me thinking. What if, the problem is with the Joomla Cookies. I did a few more Google searches and found that Joomla does not handle cookies very well it uses cookies for everything Joomla does.

*Please read the rest of this before trying the following as this configuration did not work even though it was performing the way I thought it should.

This site is on a dedicated server and no other traffic will be going to it. It is a 64-bit Quad Core Centos Server with 1G of ram. In Varnish, I added the following to the VLC file.

sub vcl_fetch {
   set beresp.ttl = 5m; # Time that the cache should be kept for.
   unset beresp.http.set-cookie; # Unset all cookies on the site
}

I “sieged” the server again at 50 requests/second and sure enough, Varnish was keeping the cache and serving it out of Varnish and not hitting apache at all. I then proceeded to stress the server at 500 requests/second and still the server was holding strong. Finally, I stressed it at 800 requests/second and it was still functioning properly. This is more than what the client would receive for traffic on an average day and I was confident that this had solved the problem….read on!

I then tried to log into Joomla on the front-end and on the administrative side. As it turns out, removing the cookies was a serious problem for the functionality of Joomla and would not let me login. I then removed the change that I made to the VLC, restarted Varnish and tried to login again. This time it worked.

I thought about how I could work around this issue, but nothing was coming to me. I did more research and came across this blog article. It is a little bit hard to read, but once you make the configuration readable, I found this code. *Note that I changed the obj.http.set-cookie to beresp.http.set-cookie to account for the new version of the Varnish 3.0 language.

sub vcl_fetch {
        set beresp.ttl = 5m;
        if( req.request != "POST" ) {
                unset beresp.http.set-cookie;
        }
}

Once I added this code to the VLC and restarted Varnish, not only could I login to Joomla on the front-end and administrative side, but the server was still holding strong at 800 requests/second.

Now, I can only stress the server to the point where I think it will hold its own in a real life scenario, but that doesn’t mean other steps may need to be implemented if the site gets picked up by Digg or Google (Google is what took down this clients server last month). Until that happens, we can’t account for the amount of traffic the site will receive. One additional step you can take is to add a Load Balancer into the mix and allow for the additional traffic and relive some of the stress on the sole server. This however, will be a separate blog entry as this one is already long enough.

In closing, I hope this article helps you to save weeks of performance testing, server changes and headaches.

I switched from Dreamhost (Shared hosting) which was great for my “own” server, but it was very restricting. I didn’t have root access and I couldn’t expand my sites performance. I moved to Linode.com and it was a great decision. If you have the skills, having full control of your server is awesome, but you need to beware of the following. If you break your server/websites/email (if app) somehow, you need to fix it. No support to call or email. Trust me on that one! Since I moved all my sites to this server, I learned very quickly about aspects of server administration that I wasn’t familiar with.

I have a configuration running CentOS, nginx and php 5.3.2 that includes svn (version control), varnish (caching), siege (stress testing), urchin (Traffic Analytics), nagios (resource monitoring) and more.

I am running all of my domains and a few of my friends websites and it works awesome. I am so excited to learn more server administration as I go so that I can be a more knowledgeable employee at my job. My work recently moved to RackSpace Cloud Servers from RackSpace Managed Hosting. I (+1) currently maintain 8 servers at work and we are adding another 8 servers over the next few months. The more I can learn on my own for selfish reasons (resume), the more I can bring to work and help them as well, which in turn will help me. Over the past 6 months,

I have learned so many exciting linux commands and processes that I feel very comfortable in command line. I know I still have A LOT to learn and ways to improve on already existing knowledge, but I am more confident now than I ever was. I added another set of skills that will greatly help my career. I am now not just a Web Developer, but a Server Admin as well.

First of all, he said “items” and I was only talking about one thing. Second, if his only way to pay you is a certified check, there is something wrong. I may be mean to telemarketers and scammers, but it is fun sometimes. This is my exact response to his reply.

My response

I wanted to setup SVN so that when I commit to the server, it automatically updates the site with the newest content. Normally you have to manually go in and update the site to the latest revision. After reading through the Dreamhost Wiki, I found out how to accomplish this in only about 5-6 quick steps. I say quick in the fact that if you do it right it is quick. I had a few issues based on false information in the wiki, but I finally got it working and they updated the article.

Read the article here.

Once I got it working, I tested it out and it worked wonders. It made my life a lot easier because not only was it one less step, but I didn’t have to go into command line after every save and commit. If you are using SVN, I would highly recommend establishing a post-commit hook so that you can save time and energy.

I wonder if Facebook even realizes the mistake or if it is just my account. Oh well, funny anyways.

I can name each transaction that I make with each account, categorize it and then tag them for later searching. It will send you emails when there is unusual spending on a specific account or merchant. It is a very robust system that I would highly recommend to anyone who has a lot of accounts.

Verizon now allows customers to manage their DVR from anywhere there is internet access. This is an amazing feature because it will allow “average” customers (not tech savvy) to go online and add/delete from the DVR while at work or on the road.

For example: Let’s say that you leave for work in the morning and at noon you find out that the President is going to be delivering a speech in a breaking news situation. You can go online and add that as one of your scheduled recordings. Don’t have room, delete other items to make space. I am not sure if it is all customers right now or just the ones with the Home Media Center plan. They might roll it out to all DVRs if the community responds well to the feature.

This is an exciting feature that I think many will like if it is marketed properly. I would want a summary of the encryption it will run and be able to see a certificate. If they can fully answer that question, I do not see a single negative aspect of this feature. Isn’t technology awesome that you can remotely control your DVR when you are nowhere near your TV. I knew it was possible, but I wasn’t sure how others would perceive it. Only time will tell if it will be available to all customers which I really hope they do.

There is also a firefox plugin for Tiny Paste which you can select the text you want in the browser, right click and convert it into a url. Very handy when you are trying to quote something and you need more than 140 characters.

Enjoy!

For example:

• There is a poster of the movie “Maverick” and someone from the Dallas Mavericks
• Inside the door on the left, there are burning books
• On the wall there are blank frames and if you click on them, she says: “Learning’s hard”, “Didn’t graduate”, “BS here I come”
• If you click on the curtain directly behind Palin, she says “Where did Russia go?”
• There is a 6 pack of beer on the seat that she calls “Joe 6 pack” and the light bulb she calls “Joe Light bulb”

I would highly recommend checking this site out for fun

I have been using it for about 1 hour and it has picked all songs that I have liked. I keep hearing about it from my friends, so I figured I would get on the bandwagon. I would highly recommend this FREE service if you are allowed to listen to stream music at work.

By the way, I won! Actually, he had me beat the entire game, but the last word in the top left corner, “HA” a triple word score and “AD” which gave me the points I needed to win.

Check it out if you have time. Scrabble Beta on Facebook

– source

When I was reading up about them, it says:
“English is our primary language. We also speak Spanish, Japanese, Swedish, Korean, Polish, Russian, Romanian, Armenian and a few other languages.” – source

That’s awesome, but out of all the other languages, Italian or German would be next on the list, I would think. It must be since they are located about 30 miles away from Glendale, CA which has the largest concentration of Armenian-American population. Just thought that was pretty funny.

Speaking of jQuery, I am heading to the jQuery Conference in Boston in the am

Zemanta also provides two other key components to this tool. Below the posting area, links will appear that relate to your content as well. You can click on the boxes that appear and see where they are coming from, for example: Wikipedia, main site, imdb, maps, youtube and more. Below those boxes in the tags section is a set of tags that are auto generated based on your content. You click the tag and it adds the tags for you.

This is a great tool because, it is very easy to be lazy while blogging and not want to link to sites or add tags, but this makes it a lot easier and faster. I would highly recommend it to any blogger, however, if you have a lot of firefox addons (depending on the addon), it might slow down your blogging because it uses all your CPU usage. I have 2Gb of memory in my laptop and it ran slow with one or more of my firefox plugins. Just informing my readers of what I experienced.

As far as I remember, you would be able to change the product key on activation on older versions of Office. Anyways, I call Microsoft asking how I can change the product key and they told me that I would have to uninstall it and reinstall it. He said that was for our protection, but I didn’t see the point in it. I would understand if it had already been activated, but it did not go through activation yet. It seems to me that someone might have more than one license of a software and may grab the wrong key code. Since Microsoft accepts that as a valid key, it continues through the installation process, but doesn’t inform you until you go to activate it that it is already active. In Microsoft’s bad attempt at making the software secure they wasted valuable time I could have spent programming. Granted it was only an hour of my time back and forth, but that’s not the point.

After I created a software plan, I needed a proposal template that could help. I went into Microsoft word and started a new document. On the left, I scrolled down to the bottom labeled “more categories” and clicked on proposals. You can choose any of them which will best suite your needs. The template is fully laid out with examples of what each section should contain. It also includes the ability to add a date picker, text fields, rich text, combo boxes and drop down options. The proposal I chose had section to mark off “Software Costs” and “Itemized Task List and Associated Hours” to make the proposal more professional.

I think Microsoft realized that if they can turn it into a PDF like controls, more people would use Office than Adobe. I will figure out more ways to use templates and content controls and learn some of the new advanced features in Microsoft Office.

Using KeepVid.com I was able to download the high quality video file (video.mp4) and imported it into Adobe Soundbooth. Soundbooth will pull out just the audio file and then you can edit it how you see fit. For me, I had to make edits to the audio and then I hit “File -> Save” and it allowed me to convert it into a mp3 so my friend could add it into PowerPoint.

Didn’t take documented steps, but it is very easy to accomplish. You will also require Adobe Soundbooth for this, however, you can use free converters to convert the mp4 into a mpg that just included the audio track. You can then import it into Windows Sound Recorder or Apple’s Garage Band. If you want to import the video, I would recommend either Final Cut Pro or iMovie for an Apple and Adobe Premiere Pro for Windows. Windows Movie Maker is alright for basic movies, but if you want to do anything advanced, you want to use one of the other programs since that is what they are designed for.

“jQuery is a fast, concise, JavaScript Library that simplifies how you traverse HTML documents, handle events, perform animations, and add Ajax interactions to your web pages. jQuery is designed to change the way that you write JavaScript.” – source

They still haven’t determined a location yet, but if you would like to attend, check out http://events.jquery.com.

The code was originally written around 1999 and contained Comma Joins.
SELECT * FROM users as u, user_details as d WHERE u.id = d.user_id
A comma separated joins are not as efficient nor are they the norm in web development. The “as” in “users as u” is not required, but that is my style.

If you have any of those joins and you move to mysql 5, you will run into some issues. You will want to convert those to INNER JOINS or LEFT JOINS, etc…

SELECT * FROM users as u INNER JOIN user_details as d ON u.id = d.user_id

Hope this helps!

*Thanks Matt for noticing the missing FROM

First, the password only has one field which you can not verify you typed it right because it is in password format. Next, they won’t let you use special characters which limits what you can increase your personal information online.

Finally, I get into the system, and it throws me (what looks like) an ASP.net JavaScript error. Since ASP is a Microsoft framework, it “only works” in Internet Explorer and I was using Firefox. When I loaded IE, it worked fine but at the same time, Firefox wouldn’t work.

1. Choose 1 player WFC
2. Choose Friends
3. Register
4. Type in your friends “Kart License Number”
5. Repeat the process on your friends end
6. Go back
7. Open a room and have friend join the room
8. Close the room and race

(amazing details I have for you!)

My friends Kevin and Kate were here as well. They came bearing “gifts” from Brighams, which was awesome. We usually get into Mario Kart Tourneys and I usually win.

So for all those Mario Kart 64 memories Kev….actually, Kate thought I should write something about her in here, so I did

They had an interesting flash page for the site and it was very interactive. However, last week, Hood legally bought Brigham’s. Friends who still work there told me that all of the “recipes” and “brands” will stay the same. I doubt that, they fired the ice cream makers. Also, All of the other workers were fired including the webmaster…see below:

If you are redoing the website because of the buyout, at least keep the so-called branding for the homepage. Step 1 to Hood ruining that which the Boston area loves.

I know this isn’t about America and it’s independence from England, but there was a lot of things going on today.

I don’t really remember to update all the time, but I will get there. I have only 8 updates total, but I was the same way with this blog. I would update every few weeks, then every few days. I have now been blogging at least once a day since December 01, 2006.

I hopefully will continue to update on a regular basis now that I have it on my Blackberry and Firefox.

After looking up the documentation we realized that if you shut down the mysql server and go into the directory that the data is located, you can type in the following command, it will fix the table without touching the data.

“myisamchk -r -q name_of_table.MYI”

Hope this helps.

The Boston Celtics are currently closing in on their 17th championship. I decided to spend the time with my parents and mainly my father instead of going into Boston which is what my original plan was.

* Update: The total for today is just shy of 4 million downloads.

Click the image for larger version.

The following is a sponsored post. “Web Hosting Pal” is a website that helps you to determine which hosting company/plan is right for you. It gives you a list of web hosting companies that have been rating in the top 10 based on price and services. Once you click on a hosting company, “Web Hosting Pal” gives you more information about them. Tells you what services they offer with the advantages of each.

You want to look for a web hosting company that gives you the best options, not just the most options. Unlimited use of the company’s music library is useless unless you are building a site for an elementary school library and music is fun to have on the kids page. For someone like me, who builds web applications, paying for that wouldn’t be worth it. However, if they offered me unlimited domain hosting to my data limit is really helpful to me, so Dreamhost works. Also for me, One click install of WordPress really helps on the different blogs I host for everyone else (not this one). I know, Dreamhost has had a few problems lately, but I am not running a high profile business off of them.

The only problem with “Web Hosting Pal”, is they haven’t updated their top 10 since October of 2005, so I’m not sure how accurate their prices are. Check out “Web Hosting Pal” and also look at Dreamhost because I really enjoy them.

If anyone has any incite on this matter, it would be awesome.