Category Archives: Development

Uploading Content to Amazon S3 with CloudBerry Labs’ S3 Explorer

2 Replies

I recently made the move to Amazon S3 and CloudFront to store and server static content, in particular images, for some of my e-commerce web sites. We have thousands of images to serve to our visitors, in all different sizes. To get started, I went to Google to do some searching for some quality tools. I stumbled upon CloudBerry Labs‘ application S3 Explorer and downloaded it to give it a try. Installation was a snap and fairly quickly, I was configuring my Amazon S3 account in S3 Explorer. What’s very cool about this is that you can store as many S3 accounts that you might have, storing them for use later on. To configure an S3 connection, you will need your Amazon Access Key and your Amazon Secret Key. Now it was time to upload!

Like I mentioned earlier, we have thousands of images. In fact, we have over 27,000 images. And that’s just in one image dimension size! We have 6 sizes, so that’s well over 160,000 images. That would be a bear to do through Amazon’s S3 web interface. Especially if I needed to set headers and permissions. CloudBerry S3 Explorer came in handy for this. I selected one set of images and before I started the upload, it allowed me to set any HTTP Headers I needed on my images. After that, up they went. I’d say with my connection, it took an hour or so to get all of them up to S3, depending on the file sizes. After uploading, I needed to set permissions, which I was able to do by just selecting all of the S3 objects and setting the proper permissions. This was kind of slow because CloudBerry S3 Explorer needed to get information on all of the objects I had selected, which was over 27,000.

All in all, I think it took me a couple of days to sporadically upload and set up all of our images. The beauty is now we’re serving them from CloudFront, which makes our sites quite a bit faster. A total win win for us.

A few notes about this wonderful application:

  • It’s incredible easy to set permissions on objects. They have a check box if you want to open the objects up for the world to download, which was nice for us. It would have been nice to be able to do this before upload like HTTP Headers, but I didn’t see how.
  • Very easy to set HTTP Headers and any meta data you need on your objects. And you can do it before the upload starts!

  • One thing that confused me a little was on Windows 7, when I minimized S3 Explorer, it went into my task bar and not with other minimized applications. It took me a little while to figure out where it was hiding. At first I just thought the application had crashed on me.
  • Overwriting object preserved HTTP Headers and permissions, something I was a little concerned about.
  • Moving data between S3 folders and buckets was really easy. Again, preserves HTTP Headers and permissions.

So, all in all, my impressions of this application are really good, and I was only using the Freeware version. The pro version, for only $39.99, offers the unlimited S3 accounts and multi-threading which speeds up your uploads. Other features available in the Pro version are:

  • Compression
  • Encryption
  • Search
  • Chunking
  • FTP Support
  • Sync

For more information on CloudBerry Labs’ S3 Explorer, check out their product page for S3 Explorer. Hopefully you’ll find this nifty little application as useful as I did!

Determine if Amazon S3 Object Exists with ASP.NET SDK

5 Replies

After my earlier posts on invalidating Amazon CloudFront objects, I thought it would be important to see if an Amazon S3 object existed before trying to invalidate it. With the 1,000 request limit on invalidation requests before Amazon charges you for them, this seemed to be a prudent thing to do. So, I turned to the Amazon Web Services ASP.NET SDK to help me out with it. This is what I came up with:

public bool S3ObjectExists(string bucket, string key)
{
    using (AmazonS3Client client = new AmazonS3Client(this._awsAccessKey, this._awsSecretKey))
    {
        GetObjectRequest request = new GetObjectRequest();
	request.BucketName = bucket;
	request.Key = key;

	try
	{
		S3Response response = client.GetObject(request);
		if (response.ResponseStream != null)
		{
			return true;
		}
	}
	catch (AmazonS3Exception)
	{
		return false;
	}
	catch (WebException)
	{
		return false;
	}
	catch (Exception)
	{
		return false;
        }
    }
    return false;
}

I decided that if I found a valid ResponseStream on the S3Response, then I had a valid object. All I’m checking on is the object key itself, i.e. an image path in S3. Another note here is I’m checking for three different exceptions but returning false for all 3. The reason I have this coded this way for now is I wanted to see what different exceptions GetObject might throw depending on what was wrong with the request. This was done purely for testing purposes and will probably be changed in the future. For instance, I discovered that AmazonS3Exception is thrown when the object isn’t there. WebException is thrown when the object is there, but the request cannot be completed. I’m still in the testing phase with this, but I hope this helps some other Amazon Web Service developers out there.

Finding Duplicate Row Values with SQL

Leave a reply

Every once in a while I have the need to find duplicate row values in a SQL table. I seem to forget how to do it each time since its not something I have a use for every day, so I thought I’d record it here and share. The solution I found was here:

http://www.petefreitag.com/item/169.cfm

Basically, the SQL code is as follows:

SELECT ColumnName, COUNT(ColumnName) AS ColumnNameCount
	FROM MyTable
GROUP BY ColumnName
HAVING (COUNT(ColumnName) > 1)

That’s it, enjoy!

Adding an Image to a PDF Document Using C# and PdfSharp

6 Replies

A while back I wrote about generating PDF documents using PdfSharp. It worked really well for us to generate invoices and purchase orders on the fly to ship orders and receive product. One must on our invoices was showing our corporate logo on our invoices that went out to our customers. We also wanted to do the same on the documents we generated for our purchase orders so that our vendors could easily identify our company when processing the order. This turned out to be a snap using PdfSharp.

void AddLogo(XGraphics gfx, PdfPage page, string imagePath, int xPosition, int yPosition)
{
    if (!File.Exists(imagePath))
    {
	throw new FileNotFoundException(String.Format("Could not find image {0}.", imagePath));
    }

    XImage xImage = XImage.FromFile(imagePath);
    gfx.DrawImage(xImage, xPosition, yPosition, xImage.PixelWidth, xImage.PixelWidth);
}


try
{
    PdfDocument doc = new PdfDocument();
    PdfPage page = doc.AddPage();
    
    XGraphics gfx = XGraphics.FromPdfPage(page);
    
    AddLogo(gfx, invoice, "pathtoimage", 0, 0);
}
catch (Exception)
{
    // Handle exception
}

The arguments to DrawImage on the XGraphics object specify an XImage object, the starting position of the image (0, 0 in this case), and the dimensions of the image. You can get the dimensions easily from the PixelWidth and PixelHeight properties (the Width and Height properties can be used, but are deprecated) of the XImage object.

Then call the AddLogo() method above to draw your image in the document. Positioning and image dimensions are important to get correct otherwise the image won’t draw properly in your document. Keep track of where you place your logo when drawing text or additional images on your document as well. If you don’t keep track of positioning from element to element, you’ll actually draw over previous elements.

E-Commerce Checkout Usability Issues

3 Replies

I develop e-commerce applications for a living and I love what I do. The best part about my job is that it gives me the chance to continue to develop software, but also be involved in actual business decisions that yield real results. During the busy holiday season leading up to Christmas mean that e-commerce web sites are some of the busiest sites on the Internet. This means that e-businesses are relying on the month between Thanksgiving and Christmas to really add to their yearly bottom line. Since its the time of year where thousands of people hit the Internet to do their Christmas shopping, and I’m no different, I thought I’d bring up an issue I ran into while doing some of my online Christmas shopping. I’m going to keep my examples really vague until after Christmas in case my wife or other family members read my blog.

The handful of websites (high profile websites actually) I visited were all fairly well designed and responded rather quickly. That’s key during a busy e-tail season. I ran into issues on most of them during checkout. One site asked me to register during checkout, which I typically do, but took me back to my cart after registration instead of letting me continue my purchase. I was sort of confused because the cart details at the top of the page were really small, so I didn’t really know what to do next. If people expect to make their purchase when they click checkout, let them continue to do so. Don’t hinder the process.

Another website I visited, like most out there, allowed me to enter an offer code during checkout. Much to my delight, they let you stack coupons. So I scoured the Internet for some coupons and entered them into their offer code page during checkout. Much to my chagrin, none of them went through. However, I didn’t get a notification that the coupon was invalid. It just removed it from the offer code section and didn’t tell me anything about what might be wrong. In the end, I couldn’t use ANY of the offers I had and actually make it through checkout. All it would have taken was to tell me the offer code was invalid. What would have been better would have been to process them all at once and tell me which codes were valid and which weren’t.

I find its really important to keep your e-commerce website as usable as possible. The checkout process is the most important part of the site and you want people in and out as quickly as possible so you can capture that conversion. If you make things difficult or don’t provide usable feedback, you may lose a customer. That can have a real impact on your bottom line.

C# Inferring SqlDbType via Reflection

Leave a reply

Today, I was looking for a way to take a string representation of a SqlDbType, i.e. System.Data.SqlDbType.VarChar and actually return the enumeration value (System.Data.SqlDbType is an enumeration of valid SQL data types like VarChar, Char, Int, etc.). Of course I could have done something like:

public static SqlDbType GetSqlDbType(string sqlDbType)
{
    switch (sqlDbType)
    {
	case "varchar":
		return SqlDbType.VarChar;
	case "char":
		return SqlDbType.Char;
	case "int":
		return SqlDbType.Int;	
	case "bigint":
		return SqlDbType.BigInt;
	default:
		throw new Exception(String.Format("{0} is either an unknown or unsupported SqlDbType.", sqlDbType));
    }
}

The drawback to this solution is that I would need a giant switch statement for all of the possible SqlDbType possibilities. Plus, what if for some magical reason, a new one is added? So I turned to Google to help me find a more elegant example, and I found what I was looking for:

SqlDbType sdt = (SqlDbType)Enum.Parse(typeof(SqlDbType), "VarChar", true);

This allows you to find the SqlDbType value you are looking for by using reflection on the SqlDbType enumeration. Nice! Of course, I wanted a more complete solution for my needs, so I came up with something like this (based on the first example):

public static SqlDbType GetSqlDbType(string sqlDbType)
{
    if (!String.IsNullOrEmpty(sqlDbType))
    {
	try
	{
	    string hintText = sqlDbType.Substring(sqlDbType.LastIndexOf(".") + 1);

	    SqlDbType sdt = (SqlDbType)Enum.Parse(typeof(SqlDbType), hintText, true);
	    return sdt;
	}
	catch (ArgumentException)
	{
	    throw new Exception(String.Format("{0} is an unknown SqlDbType.", sqlDbType));
	}
	catch (Exception e)
	{
	    throw e;
	}
    }

    throw new Exception("Provided value for SqlDbType was null or empty.");
}

This lets us do a couple of important things. First, it allows for support of the complete namespace for a SqlDbType, i.e. System.Data.SqlDbType.VarChar AND just VarChar. Second, it lets us catch important errors like an unknown SqlDbType value that is passed in to GetSqlDbType.

That’s it! Enjoy! If anyone has any questions or comments, you know what to do…

Invalidating Content on Amazon CloudFront with ASP.NET SDK

2 Replies

I’m working on integrating some of Amazon’s Web Services into out eCommerce platform. I’ve been working on performance enhancements on and off for the last year and content delivery is the last big step for us. Getting started on S3 and CloudFront was pretty easy, but I ran into some issues when updating content in our S3 buckets. Luckily, Amazon added the ability to do this at the end of August. Since we use ASP.NET, I’ve started to work with their .NET SDK. Turns out, its pretty easy to invalidate some content.

public bool InvalidateContent(string distributionId, List<string> files)
{
    AmazonCloudFrontClient client = new AmazonCloudFrontClient(Settings.AWSAccessKey, Settings.AWSSecretKey);

    PostInvalidationRequest request = new PostInvalidationRequest();
    InvalidationBatch batch = new InvalidationBatch();

    batch.Paths = files;
    batch.CallerReference = GetHttpDate();

    request.InvalidationBatch = batch;
    request.DistributionId = distributionId;

    PostInvalidationResponse response = client.PostInvalidation(request);
    if (!String.IsNullOrEmpty(response.RequestId))
    {
	bool bInProgress = true;
	while (bInProgress)
	{
	    GetInvalidationRequest getReq = new GetInvalidationRequest();
	    getReq.DistributionId = distributionId;
	    getReq.InvalidationId = response.Id;

	    GetInvalidationResponse getRes = client.GetInvalidation(getReq);
	    bInProgress = getRes.Status.Equals("InProgress");

	    if (bInProgress)
	    {
		Thread.Sleep(Settings.AmazonWaitTime);
	    }
	}
    }
    else
    {
	return false;
    }

    return true;
}

InvalidateContent expects a CloudFront Distribution ID and a list of S3 files to invalidate. References to the static Settings class is just a class that reads in configuration settings from a configuration file, be it App.config or any configuration file you wish to set up. Basic values to store are your AWS Access Key, AWS Settings Key, and a wait time value before requesting information from Amazon again. There is also a method GetHttpDate(), which just returns:

System.DateTime.UtcNow.ToString("ddd, dd MMM yyyy HH:mm:ss ", System.Globalization.CultureInfo.InvariantCulture) + "GMT";

InvalidateContent() will make a PostInvalidationRequest (which is part of the ASP.NET AWS SDK) through an AmazonCloudFrontClient object. If this request is successfully posted, Amazon will return you a RequestId value that you can use to poll for the completion of your request. Keep in mind that you can only invalidate 1,000 S3 objects a month. After that Amazon will start to charge you $0.005 per object invalidated. This is per file, not per invalidation request. Hopefully you found this helpful!

ASP.NET SQLCacheDependency with SQL Query Notifications

Leave a reply

I’m going to make this quick and dirty. I’ve set up my ASP.NET web applications to leverage SQLCacheDependency with SQL Server 2005 Query Notifications. The setup process takes a some time and debugging can be tricky. But the payoff can be enormous. The bottom line is that the performance increase on page load times is well worth the effort, especially for data that doesn’t change all that often. I found it really useful for caching product data on my eCommerce sites, especially as the number of products in the system grew to over 5,000. However, I always seem to miss a step when when configuring my SQL Server 2005 databases; so this post is for my own reference, but if it helps someone else out there, even better.

The original article I used as a basis for configuring my applications is at the url below:

http://www.simple-talk.com/sql/t-sql-programming/using-and-monitoring-sql-2005-query-notification/

Follow the steps there and you’re good to go. Especially use the SQL Profiler debug steps at the bottom of the article if you get tripped up. One thing that I always had to do with my databases to get everything to work properly was execute the following query:

use <dbname>
EXEC dbo.sp_changedbowner @loginame = N'<dbuser>', @map = false

Make sure you use this caching technique responsible though. The Query Notifications can use up processing power so only cache data that you know will give your application a performance bump. Also beware of memory usage as you cache more and more data. You could end up caching so much data your application needs to restart often and that could cause slow page load times.

Easily Cause StackOverflow Exception in ASP.NET Web Application

Leave a reply

This is a short one, but since I can’t believe I’ve never managed to do this before, I thought I’d post a little tidbit on it. I threw a StackOverflow in one of my ASP.NET C# web applications today. Watching the request keep going for about 5 minutes, I checked the Event Log on the server. There it was, StackOverflow exception. Say, what?

How did I manage it? Recursively add a control to its own control collection. The .NET framework just freaks and the application pool just gets restarted after each exception kills the app. Glad it didn’t take long to figure it out!

Properties In a Active Directory Computer Object

Leave a reply

I’m working on a project and I was curious about what properties Active Directory would return when search for computers with the DirectorySearcher class in C#. Basically, you can search an entire domain for known computers using something like:

string domainName = "mydomain";
DirectoryEntry de = new DirectoryEntry();
de.Path = String.Format("LDAP://{0}", domainName);
de.Username = "username";
de.Password = "password";

try
{
     DirectorySearcher ds = new DirectorySearcher();
     ds.Filter = "(&ObjectCategory=computer)";
     SearchResultCollection results = ds.FindAll();

     foreach (SearchResult sr in results)
     {
             ResultPropertyCollection myResultPropColl;
             myResultPropColl = sr.Properties;
             foreach (string myKey in myResultPropColl.PropertyNames)
             {
                 string propString = String.Empty;
                 string tab = "    ";                                
                 Console.WriteLine("{0} = ", myKey);
                 foreach (Object myCollection in myResultPropColl[myKey])
                 {
                     Console.WriteLine("{0}{1}", tab, myCollection);
                 }
         }
     }
}
catch (Exception)
{
}

This code is basically borrowed from this MSDN article, but what I was originally after was what properties could I actually get at? I wasn’t all that interested in the code. Well, just running that code yielded me the following list of properties that you can access via a computer object from Active Directory:

operatingsystem
countrycode
cn
lastlogoff
dscorepropagationdata
usncreated
objectguid
iscriticalsystemobject
serviceprincipalname
whenchanged
localpolicyflags
accountexpires
primarygroupid
badpwdcount
objectclass
instancetype
objectcategory
whencreated
lastlogon
useraccountcontrol
samaccountname
operatingsystemversion
samaccounttype
adspath
serverreferencebl
dnshostname
pwdlastset
ridsetreferences
logoncount
codepage
name
usnchanged
badpasswordtime
objectsid
distinguishedname

For my purposes, I wanted to find out operating system information, so the operatingsystem and operatingsystemversion properties woulded nicely for me. But, there you go, all of the properties you can access from a computer object in Active Directory. I hope this is useful to someone else out there other than myself.