Category Archives: Web Development

Cranked stuff about Web Development

Easily Cause StackOverflow Exception in ASP.NET Web Application

Leave a reply

This is a short one, but since I can’t believe I’ve never managed to do this before, I thought I’d post a little tidbit on it. I threw a StackOverflow in one of my ASP.NET C# web applications today. Watching the request keep going for about 5 minutes, I checked the Event Log on the server. There it was, StackOverflow exception. Say, what?

How did I manage it? Recursively add a control to its own control collection. The .NET framework just freaks and the application pool just gets restarted after each exception kills the app. Glad it didn’t take long to figure it out!

Remove ViewState From ASP.NET Page to Improve Performance

Leave a reply

In a previous post, I eluded to the fact that I remove the ViewState from our ASP.NET pages to improve performance. I can’t take credit for coming up with the idea though. Originally, I got the idea and solution I wanted to implement by reading this article at EggHead Cafe. The solution I chose was to store the ViewState on the web server’s file system and reference the file in the page sent back to the client. I’ll outline how I did that below.

The first thing you’ll want to do is create a new Page base class that inherits from System.Web.UI.Page so you can override some of its methods, namely SavePageStateToPersistenceMedium and LoadPageStateFromPersistenceMedium. This will allow you to save the page’s ViewState to the file system and then re-load it on a post back. Let’s start with saving the ViewState to the file system.

protected override void protectedSavePageStateToPersistenceMedium(object viewState)
{
	// Serialize the view state into a base-64 encoded string
        LosFormatter los = new LosFormatter();
	StringWriter writer = new StringWriter();
	los.Serialize(writer, viewState);

	string guid = NewGuid();
	string vsp = ViewStateFilePath(guid);

	// Save the string to disk
	StreamWriter sw = File.CreateText(vsp);
	sw.Write(writer.ToString());
	sw.Close();

	// Save the guid in a form field
	ClientScript.RegisterHiddenField("__VIEWSTATE_GUID", guid);
}

So, let’s step through what we’re doing here. The first few lines of code, we’re serializing the view state. Next, where we call

string uid = NewGuid();

We’re creating a new guid that we will use in creating the file name on the server for the actual value of the current view state. NewGuid() just returns a new guid value from the System.Guid class.

Next, we need to create a path to where we’re going to store the file as well as its file name. Now, you can do this any way you want as long as all of your files end up being unique. You can’t go overwriting one guy’s view state with someone else’s. Now I based mine on basically the guid appended with the current request path minus the extension and replacing the slashes with dashes. So, my filename looks like:

string fileName = guid + "-" + Path.GetFileNameWithoutExtension(Request.Path).Replace(
    "/", "-") + ".vs";

Where the guid was passed in to the function that I called to create the view state file path.

So now that we have a path to where we can write the file, we can go ahead and do so using the StreamWriter that we created. Now, the last thing to do is spit out where we can find the view state to the client. This is done by registering a hidden field with the client:

ClientScript.RegisterHiddenField("__VIEWSTATE_GUID", guid);

That GUID allows you to recall the proper file for the client when you need to access the ViewState.

So now you’ve persisted the view state to the file system. Now all that’s left is to load it up when needed, which is done with LoadPageStateFromPersistenceMedium(), which is below.

protected override object LoadPageStateFromPersistenceMedium()
{
	string guid = Request.Form["__VIEWSTATE_GUID"];
	if (guid == "")
		return null;
		// determine the file to access
	if (!File.Exists(ViewStateFilePath(guid)))
	{
		return null;
	}
	else
	{
		// open the file
		StreamReader sr = File.OpenText(ViewStateFilePath(guid));
		string viewStateString = sr.ReadToEnd();
		sr.Close();
		// deserialize the string
		LosFormatter los = new LosFormatter();
		return los.Deserialize(viewStateString);
	}
}

This approach gives you the ability to minimize the amount of data that you have to send over the wire to a client browser. There are other solutions to this like storing the ViewState in memory, but I felt that it was a waste of resources if a ViewState might only be used once and then abandoned. The one thing you’ll want to do is manage your saved ViewState files on your disk. These files can get out of control, especially when you get a lot of visitor’s. So what I did was just set up a Scheduled Task in Windows to clean up stored ViewState’s every three hours. This works out pretty good since a ViewState really won’t be needed beyond that timeframe.

Anyway, I hope this solution helps some other C# ASP.NET developers out there. It sure worked out great for me!

Migrating a pMachine Website to WordPress

Leave a reply

Anyone who has used or is still using pMachine for their website and forum knows that its outdated and a pain to use. Today, there are tons of alternatives to content management, blog management, and forum management that you can find the best tool that works for you. A few years back, EllisLab replaced pMachine with ExpressionEngine, which from the little that I’ve used it and heard about others using it, is way better than pMachine. But, being a WordPress fan, I wanted to use that instead of ExpressionEngine. That and WordPress is free, which is a plus. The WordPress developers have also come up with a great forum engine, called bbPress, so I decided that would be a great replacement for the pMachine forum.

So now I know what software I want to use, but the first thing I had to figure out was how easy would it be to migrate our pMachine data over to WordPress. No good importer for pMachine comes with WordPress, so out to my good friend Google I went for answers. Pretty quickly I found two articles about migrating pMachine to WordPress 1.5 here with some actual PHP code to import the pMachine blog here. So, I was off to a good start.

Since bbPress can integrate directly with WordPress, I needed to use WordPress version 2.5 so that the cookies between the two applications would work. Apparently WordPress changed their cookie implementation in newer versions, so until bbPress gets an update, we’re stuck with the old WordPress version. There aren’t any known security risks with WordPress 2.5, so we’ll be ok. I installed both applications and then went to importing my pMachine data.

Based on the PHP scripts I downloaded from the article mentioned above, I ended up writing a bunch of my own PHP scripts to import all of our user data, post data, comment data, and forum data into WordPress 2.5 and bbPress. I didn’t want to install WordPress 1.5 first, so I took the code snippets I needed and went to work.

It took the better part of a week to import and test everything out and make sure all of the proper relationships between users, posts, and comments were set up properly. I’m currently packaging the PHP scripts up to release to the world under some public license so everyone else can who was in my position will have a solution. I also want to get the final version of GhostDroppings up and running and make sure my users are happy with the migrated data, but so far everything looks good. So check back in a bit to download the code!

ASP.NET Site whitehouse.gov Reviewed

Leave a reply

So I was over on Slashdot last night looking for something interesting to read and ran across this tidbit about the new whitehouse.gov site that runs on ASP.NET. Honestly I think the only reason this got mentioned on Slashdot is that yesterday was Inauguration Day. Any other day and it probably would have fallen through the cracks. Anyway, I decided to take a look at the pointers that the author brought up to see if there was something I could learn. Most of the improvements I already knew about, but there were a couple that were new to me. Two that I’d like to consider implementing in my sites are:

  • Remove X-Aspnet-Version: header (remove 30 bytes per request)
  • Use compressed JQuery from Google’s servers (lower latency and improve performance)

The one issue with using JQuery from Google is your tied to whatever they’re using. If a newer version comes out and you want to use it but Google doesn’t, you couldn’t do this. Luckily I use JQuery 1.2.6, so this isn’t an issue for me.

As for the author’s review, I thought it was pretty good. A real world example is always a great illustration of what to do and what not to do. I’d have liked to see him make a suggestion on how to fix one of the issues he found, the ViewState issue, which would have been useful for other developers making the same mistake.

ViewState is still necessary for an ASP.NET application, however you don’t have to pass the entire ViewState back to the client. Its a waste of bandwidth (and can ’cause some nasty Base64 exceptions). One solution I use is to store the ViewState on the file system and only pass the file name back to the client for reference later on. It takes up a lot less bandwidth than a potentially huge ViewState. Other solutions are storing it in memory, the database, or by some other means. We clean out our old ViewState’s every 3-4 hours as many of them aren’t needed after that point (I’m thinking this might make a good article in the future).

Another example that kind of irked me was the site’s use of meta keywords. Uhm, yeah, this might not be as relevant anymore for Search Engine Optimization, but its still not a bad thing to have it in there. Just keep it to 200-300 characters. Nothing too crazy.

One last thing that he pointed out that I just didn’t agree with was that the server was running IIS6.0. Now, correct me if I’m wrong, but IIS7.0 is only supported on Windows Server 2008, right? Well, IT departments have budgets and all, so maybe a Windows Server 2008 license or the hardware to install it to wasn’t available? I know in my case, the budget doesn’t always allow for the latest and greatest, even if I want to use it. So to knock the development team for using IIS6.0 seems a little over the top if you ask me.

This entire site could definitely use some improvements, which the article nicely points out. To go one step further, I suggest any web developer install YSlow for Firebug for the Firefox browser. This came in handy for me when I was trying to optimize my sites. The whitehouse.gov site’s YSlow score is a 51 (an F), which is horrible. When I started using YSlow, I noticed some of my sites had a similar score, to which I was appalled and went to fixing pronto. By implementing some of the changes that I could as suggested by YSlow, I got us up to a 78, which is a high C. Some changes you can’t make (like Javascript at the bottom of pages and using a Content Delivery Network) due to how your application works and changing them just to make a minor improvement is more trouble than its worth. However, there isn’t any excuse to have an ASP.NET site that scores so low. Those folks over at whitehouse.gov definitely need to clean things up!

URL Rewriting in ASP.NET – ISAPI Rewrite vs. UrlRewritingNet

Leave a reply

Seeing your web site rank well in the major search engines (Google, Yahoo, MSN) is something that every web developer strives for. Making sure your web site’s pages are search engine friendly is a huge part of that effort. With ASP.NET, pages typically end in the .aspx extension. While this isn’t bad for SEO, most ASP.NET pages are also dynamic. So having a dynamic page that is rendered based on a bunch of query string variables doesn’t get you anywhere with SEO, especially if you use a lot of them. So for a while now, we’ve used ISAPI Rewrite to rewrite your dynamic pages into something that is a lot more user friendly. This ISAPI extension isn’t free though. It costs $99 to purchase a license. Not that bad, right?

Well recently, I discovered an Open Source solution called UrlRewritingNet. It was originally developed in 2006 for the 2.0 framework, however the developers claim it will work all the way up to the 3.5 framework. I’m not sure how much further development is being done though, as the last release was in August of 2006. It is Open Source, so theoretically you can download the source and make modifications yoruself if you need to.

UrlRewritingNet integrates directly into your ASP.NET web application as a HttpModule. With each incoming request, this module is called to see if the URL requested is a rewritten URL. This isn’t too different from ISAPI Rewrite except that the rewrite for ISAPI Rewrite is handled higher up the stack by IIS itself and not the web application. Using one solution over the other shouldn’t be much different from a performance standpoint, but tests would need run to prove it. That’s a task for another time however.

The one drawback, for me anyway, was that I did see that UrlRewritingNet requires all of the rewrites to be defined in the Web.config file. I’m not a huge fan of that. Web.config gets cluttered up enough as it is, so the more rewrites you need the bigger this file is going to get. With ISAPI Rewrite, all of the rewrites are stored in a separate file called httpd.ini, much like rewrites in Apache. On the plus side, it looks like you can extend UrlRewritingNet by developing your own rewrite rule providers. So if you need some functionality that isn’t provided, you can hook what you need up yourself.

UrlRewritingNet looks pretty promising and I hope I have some time to check it out and see if it is a good substitute for ISAPI Rewrite.

Minimizing Downtime When Deploying ASP.NET Web Applications

2 Replies

One of the annoying things I find with managing our ASP.NET web applications is that I need to “shut down” our sites when deploying a new version of our assemblies. Its not that I’m worried about visitors seeing our maintenance page for 20-30 minutes, but as search engines spider our site during this down time, they can’t find pages that they knew about before. This is especially annoying when analyzing my sites in Google’s Webmaster Tools and see that I have a bunch of HTTP errors because Google couldn’t find a page because the site was unavailable. Since Google puts a high regard on site availability and quality when determining rankings, I’d like to avoid this.

Deploying the site is actually very simple. We use the XCOPY method of pushing up web forms, controls, and assemblies. But if you just start overwriting files in the live site, users get errors or inconsistent pages. And, if any database changes need to be made, code could not function properly before updating the site. Any of these problems would affect my Google issue I mentioned above as well. Not only that, but any developer worth his/her paycheck tests their changes in the live environment anyway. So just tossing up the new version is no good.

I’ve been considering a few different solutions to this, but I haven’t come up with something that solves the issue completely. At some point, I still have to take down the site.

One solution that I thought of was that I could set up a staging server to deploy my changes to and then run my tests there. Once I’m satisfied with the results, I could push it to my live site, minimizing the amount of down time. I figure the max amount of downtime using this approach would be 5-10 minutes depending on if I had to make any database changes. Not a perfect solution, but better than the 20-30 minutes I’m experiencing now.

Another solution I thought of was to set up a web farm. I could deploy the new version to one web server, make sure everything is good to go, then deploy it to the second web server. Users could still visit the site because at least one server in the farm could handle the incoming request. But this wouldn’t work great if database changes needed to be made. The site itself would still have to come down.

So right now, solution #1 appears to be the best approach and easiest to implement. Maybe I’m making more of a big deal of this than I need to, but I think everyone wants to minimize the downtime of their site. The one reason I’m holding back on changing my approach is I don’t know how much Google or other search engines weigh not being able to access a site for a short period of time. Regardless, I’m curious what other solutions developers and web teams use to deploy their ASP.NET applications to minimize site downtime. I’m positive there is a solution to my problem, but I just haven’t thought of it yet. If anyone has something that works for them, please chime in here!

Nextopia Search Integration

4 Replies

We’re on our third search integration in the last few years. We used to use SLI Systems for search. They have a phenomenal service for search, but its a little more expensive than our small company can afford. So, last year, we decided to switch to the Google Mini. We had heard some good feedback from some other sites that had implemented it and decided to go for it. The cost for us was only $3,000 for the Google Mini device, the $200 a month co-location fee we pay, and some development time.

In the end, the Google Mini just didn’t give us consistent results for the how we were using search. We were able to implement a pretty cool solution that let you refine results by price and category, something I hadn’t seen anywhere else. But at the end of the day, not being able to get a result for something as simple as an item number was just too frustrating. I tried a lot of different techniques with the Mini to get the results to come back properly, but at the end of the day, it just didn’t cut it. I’m not sure if its because of the fact that it depended on spidering the site or not that gave us issues, but we had to start looking at other solutions.

My boss sent me an article about three different search solutions and compared them for readers. After reading, we decided to give Nextopia a shot. SLI was one of the other choices, and as much as I liked their service, we just still couldn’t afford it. The reasoning for trying Nextopia was that it provided similar features that SLI had and the price was way more appealing. You can get search for as little $995 a year. We’re paying a bit more than that for our three sites that get decent traffic, but its still affordable for our small company.

Nextopia provides you with a search page and template, but it didn’t fit in with the look and feel of our site. Since I wasn’t happy with the code they wrote for it either, it was more work to modify their version than to roll my own. Luckily, the code we had for our SLI and Google Mini implementations was written well enough that I was able to abstract that code even more (had to refactor a bunch of it) and roll a Nextopia version of our pre-existing search pages. What’s nice is now we have a C# library with 3 search integrations and the ability to roll new ones at any time we want.

The integration process took about a days worth of development and testing, which isn’t bad for integrating a 3rd party search solution. Granted, the refactoring of our search library took more time, but the actual Nextopia integration took a day. We’re really happy with the results it can provide and how quickly it can provide them. This definitely gives our users the robust search solution they need to find out products reliably and quickly. I’d say the only downside to the integration though is I can’t automatically send them a feed of our products. I have to update it manually on a regular basis, which is kind of annoying but not the end of the world. Anyway, if you’re looking for an e-commerce search solution, I highly recommend Nextopia.

Keyword Tracking with Caphyon’s Advanced Web Ranking

Leave a reply

I’ve been honing my skills over the last few months with optimizing our e-commerce sites for search engines. Its been a long, arduous task. One that requires the utmost patience. When things go wrong, it seems like the world is ending. But, like one of my favorite books was famous for saying, Don’t Panic!

Over this period of time of getting up to speed on search engine optimization, I’ve been looking for a tool to help me track our rankings for keywords we’re going after. I had been using this huge Excel spreadsheet to track rankings on a bi-weekly basis, but it was a pain in the butt to update. I found some online tools that would let you track a few keywords, but nothing all that awesome. That, and I really wanted a desktop application for this. Don’t ask me why.

So, I was reading YOUMoz the other day and came across this post about the author’s troubles with MSN Live Search. She mentioned she used this tool to track keyword positioning, so I decided it was worth a look. I downloaded the 30 day trial version and installed it on my MacBook Pro.

Setting it all up with keywords I was interested in for my site was a little time consuming, but I didn’t mind that. I gave it a run and I got a great breakdown of keywords we ranked well for and keywords we didn’t. This is cool as now I can see where I need to put some work in. But that wasn’t the coolest part. The following day, I ran it again. Now I could compare my results from today to the previous day. Kick. Ass. Even better, it stores your results for each day, so you could see how you did over the course of a month or a quarter or a year.

I find this to be a huge time saver because I don’t have to manage my rankings. All I have to do is add new keywords I want to track. When I want to update my positioning results, I just click a “Play” button and away the tool goes. I highly recommend this application to anyone tracking their SERPs in any of the major search engines.

Has Google Tweaked Their Search Algorithm?

Leave a reply

In mid to late November, I noticed traffic to one of my e-commerce sites dropped dramatically. Some quick research in Google Analytics turned up that our Google organic traffic dropped off significantly. This came as a huge shock to us because we were cruising along really, really well for some important key terms that drive quality traffic to our sites. This equates to lost dollars since we aren’t getting the visits that could turn into conversions.

After some research and conversations with other web developers I know, I came to the conclusion that nothing I had really done to our site would have caused the drop off in traffic. What’s worse is that we seemed to have dropped out of the Top 100 for key terms that we had previously ranked high in the Top 10 for. Really. Big. Problem.

So what happened? My guess is Google tweaked up their search algorithm a little. My reasoning came from the Google tools that I use on a daily basis to manage my sites. Take a look first at the image below. What it shows is our Google organic traffic from November 1, 2008 through December 4, 2008.

What you can see here is that we were flying at something around 1,800 to 2,300 visits a day. Good stuff. Then around November 11, things dropped down sharply. I kind of panicked here, but didn’t change the site up any. Then on the 14th, we were back up again. Way up. For the next 5 days, we were up around 2,600 visits a day, even peaking over 2,700 one day. Then on the 15th, things fell apart again. We were down slightly. Ok, only one day. Don’t panic.

Then I did panic. On November 20th, we were down to just over 1,000 visits. Ouch. Over the next 2 weeks, we went as low as around 850 visits. This was a problem. What happened? I didn’t change up the site in a way that should have affected our rankings from all the research I had done on my own and with other developers. So what gives? Take a look at the image below, which shows Google crawl stats

See the huge spike around the second week in November? That’s about when things started to hit the proverbial fan. This spike looks like a deep crawl of our site by Google. I’ve never seen Google crawl our site this much all at one point. To me, this indicates changes were afoot over at Google. What changes, who knows. Only Google does. What I do hope is that our site comes back up in the rankings as its a well built site, adhering to Google’s guide for building quality sites.

The bright spot here is that we’re starting to come back for some of our key terms and our organic traffic from Google is looking better. At the same time, our traffic from MSN and Yahoo has continued to get better, so this is another indication that Google changed something up. Also, using Google Webmaster Tools and Google Analytics can provide you with great insight into what Google sees of your web site as well as how people find and use your site. I highly recommend using both if you don’t already.

Google AdWords Now Has Device Platform Setting

1 Reply

I just set up a couple of new campaigns in Google AdWords this afternoon and, for the first time, noticed that you can now select a device platform for your ad campaigns. The choices are desktop or laptop computers, or mobile devices that support a full web browser. These mobile devices are devices like Apple’s iPhone. Click the screen shot below to see these settings:

This is definitely interesting because there are surely some instances where you’d want to restrict your ads from showing up on a mobile device. I’m considering turning it off for our campaigns because we run an e-commerce site and its really unclear at this point if people are really into shopping online using their mobile devices. I have no doubt that this will become widespread behavior in the future, but with mobile devices with full browser capability just coming to market, I have a feeling this is a small niche crowd that would shop via their mobile device.