Jul032008

Broken Window, Software Entropy

I am reading The Pragmatic Programmer, From Journeyman To Master. It seems to be a nice book. In the first chapter it is talking about software entropy. Basically if the software has some parts that are buggy (of course you dont write code that has bugs), it will spread out to other parts too till the whole software is rot.

According to the book, Broken Window is a theorem, that New York Police Department used to clean the streets. Basically the idea is, if there is a street and some of the windows are broken, and you don fix those, people start littering. They start thinking that the street is abandoned, and not secure any more, so if u have broken windows left unrepaired, u ll see more crime, graffitis etc. How does this apply to software? If you have buggy codes left in your application, it wont be secure any more, and it will be more and more buggy, crime is going to go up :). 

If you have a bug in the code, fix it. I know rapid shipping sometimes will stop you fixing all the bugs, but then remove the feature that your bug is in. Dont let broken windows to be spreaded around, dont let other developers (including urself) think that your code is abandoned.

Have fun coding



Tags:

E-mail | Permalink | Trackback | Post RSSRSS comment feed 0 Responses

Jun302008

Use Cache[] not Application[]

When i was checking one of my colleague code, i saw that he is using Application[] to hold some values. This is ok, you might be in need of this; however my friend was using this to cache the datatable that returns from the sql server. I asked him, "Why dont use Cache?", guess what he said: "This is my cache, what are you talking about!!". I think he wasnt aware of Cache class in asp.net. I explained him a little bit of the details, and he changed his code. You can use HttpContext.Current.Cache to cache some data, and retrieve them very easily. Not only the ease of usability, but there is also some extra features like dependencies, cache life time etc. For example, you can bind this cache to a sql server table, and the cache mechanism will invalidate the cache when the table changes. Besides putting sql server cache dependency, you can also have cache lifetime, which will invalidate the cache after the time you specified.

An example of caching the datatable is below:

[code:c#]

DataTable dtUsers = new DataTable();  
if (HttpContext.Current.Cache["PhoneBook"] == null)  
{   
     ... get the dtUser from the database    
     HttpContent.Current.Cache.Add("PhoneBook",dtUsers,null,DateTime.Now.AddHours(1),
        TimeSpan.FromMinutes(5),System.Web.Caching.CacheItemPriority.Normal,null);  
}  
else    
    dtUsers = (DataTable)Cache["PhoneBook"];

[/code] 

In the above code, i didnt put any sql server dependency, however i put a cache life time 1 hours,also sliding of 5 minutes.

Why didnt i use Web.Cache? For some reason, System.Web.Cache seems to be working per user, i dont know why?, but when i used HttpContext.Current.Cache everything seems to be working fine. 



Tags: ,

E-mail | Permalink | Trackback | Post RSSRSS comment feed 0 Responses

Jun242008

I passed 70-528

Today i passed 70-528 Microsoft .NET Framework 2.0 - Web-Based Client Development. this is my second certification exam after sql server.

keep taking the exams




Tags:

E-mail | Permalink | Trackback | Post RSSRSS comment feed 0 Responses

Jun232008

New Book: Pragmartic Programmer

I started to read a new book a few days ago; The Pragmatic Programmer, From Journeyman to Master by Andew Hunt, David Thomas. so far i am enjoying the book, once i am done with i will write a brief review about it.

 



Tags: ,

E-mail | Permalink | Trackback | Post RSSRSS comment feed 0 Responses

Jun202008

Optimizing code at work

Our campus site is using an iframe, and we encourage the other sites on campus using our iframe to have an identity of the university. This iframe has links to campus address, campus online directory, contact campus page link, an index of the campus pages, google search and most importanty a list box of quick links. This quicklinks informatin is held in the database, and unfortunately when it was programmed, nobody thought of performance, or optimizing i guess. The page accesses the database everytime iframe is loaded. Our main campus page gets a hit of 25000 to 35000 hits per day, and as we have decentralized structure, we cant control other departments' web site, so we have no idea how many hit they get, but roughly i will say, the iframe gets a hit of 45000 a day. Guess what, as there is no caching, or any other optimizing structure, the web sites hit the database 45000 times a day just to get a 80 or 85 rows of data which is quicklinks. It is not easy to add or delete a link from the quicklinks as it requires some meetings, management decisions so i can say it is mostly read-pnly database; it has been updated once a month at most.

So why was caching the information not thought at the beginning; i have no idea; but whenever i see code that i think it is really ugly, i try to fix it. What is the best fix for this? I can think of some different approaches:

  • Read the quicklinks from the database, put it in a cache, have sqlcachedependency to monitor the table (I will call this Approach I); so whenever there is an update dont read the dirty cache but update the cache
  • Read the quicklinks from the database, put it in a cache; have 1 day of lifetime for cache (Approach II)
  • Dont use database at all,use an xml file read; from the xml file, so dont hit the databas; however it requires a disk access of 45000 times (Approach III)
  • Dont use a database at all, use an xml file, read from the xml file one time during a day. so if there is an update, it will be reflected to quicklinks the next day (Approach IV)
  • Like Approach IV but instead of reading the xml file the next day, read the last modified day, and if it is different from the last read day, read the xml file again (Approch V)
  • Hardcode the links in the code :) (Approach VI)

Immidiately i eliminate Approach VI, as i dont like putting data in the code itself. Approach I looks ok to me at the beginning but we are only talking about 80 rows, and the links are updated once a month, i dont think it is good idea to poll the table for new updates, so i eliminate this. Approach II  too is basically like Approach I, but we dont poll the database for updates, we just read the entire table (once again we are talking about 80 rows at most) the next day, this is better  than approach i, as there will be 1 database hit per day. Comparing to 45000 hits per day, this of course a lot better approach, and if there is any update in the table, the worst case it will be up in the site after a day.

Approach III is eliminated as it hits the disk 45000 times. Approach IV is of course better, as it will hit the disk 1 time per day to read the xml to the memory. I dont see any performance implementing approach V against IV cause, the xml file will be very small, and instead of checking the last modified date, the whole xml could be read easily. 

So what is the verdict and why ? I picked approach II, as it hits the database only 1 time a day, and use cache to store the data. If there is any update in the table, it will be reflected to the site the next day. In fact i was planning to implement approach IV, as there wont be any database hit, and reading 80 lines of data in an xml will be faster, but we might be sharing or reusing this quicklinks table in other projects too; so instead of sharing xml file between projects and ending up different versions of the xml file in the future, i decided the approach II. 

Have fun refactoring :)



Tags:

E-mail | Permalink | Trackback | Post RSSRSS comment feed 0 Responses

Jun182008

Use your IDE (Visual Studio)

After i have Resharper (thanx to the inland empire .net user group) i understand that IDE can really help you write better code. Visual studio is a very powerful ide and i dont think i, myself utilize it too much, i want to a list of nice things i like with visual studio.

  1. Line numbering, i love to see line number in the code file so whenever the error message is talking about a line number, i can easily navigate to that line, how do you activate it for all your codes ? Go to Tools => Options => Text Editor => All Languages => Check Line Numbers section
  2. When you are writing the code, and you refer to a .net class but dont have the using statement ? Hit Control+.
  3. Your IDE's default language setting has changed from C# to VB.NET (or anything else?) ? Tools =>Import and Export Settings => Reset All Settings => No, just reset... => Select your language
  4. You need to come back to a certain line of code? put a bookmark,Edit=>Toogle Bookmarks r Control K,Control K , to go to that line Control K+N
  5. You want to leave a note to urself or to a friend who is going to maintain the same project? Comments to too difficult to manage task. Just add a comment that starts with //TODO: and when you compile the code, you will see the note in the tasks window. 
  6. TODO: isnt good enough? you want to add your own types? Tools=> Options =>Environment => Task List. You can add yours tasks types here giving them a priority too
Do you have any more ?


Tags:

E-mail | Permalink | Trackback | Post RSSRSS comment feed 1 Responses

Jun172008

Refactor your code

As much as i am trying to learn and implement test driven, we have lots of projects in our hands that dont have any unit tests. Writing unit tests to the preexistingg functions are'nt that easy. One of the reasons is most of the functions are heavy-duty workers, they do lots of stuff. Unit testing or at least thinking how to test your functions teaches you to write very simple, short functions; and if you already have heavy duty functions, u try to refactor them. By refactoring, u usually endup having very small functions; a few lines maybe, and also having more independency; if you think about the scraper functions i posted a few days ago, one of them: GetSource was a heavy, obese function. It has more than 1 type of dish on its plate to eat, so let's try to refactor that code, below is the code before refactoring

[code:c#]

 public string GetSource(string Url)
{
if (Url == String.Empty)
{
this.htmlSource = String.Empty;
return String.Empty;
}

string htmlSource = String.Empty;

if (!(Url.Contains("http")))
Url = "http://" + Url;

try
{

//create a web request
WebRequest request = WebRequest.Create(Url);
WebResponse response = request.GetResponse();
Stream responseStream = response.GetResponseStream();
//himm encoding ? do we need this if everything is english? dunnp
Encoding utf8Encode = Encoding.GetEncoding("utf-8");
StreamReader readStream = new StreamReader(responseStream, utf8Encode);
htmlSource = readStream.ReadToEnd();
}
catch
{
htmlSource = String.Empty;
}
this.htmlSource = htmlSource.ToLower();
return htmlSource;
}

[/code] 

 The first few lines of code is checking if string is empty, if not, does it have http:// prefix or not. So lets refactor this part, and write a function, below is the small function

[code:c#]

public string CheckHttpPrefix(string url)
{
     if(url.Trim() == String.Empty) return String.Empty;
     if(url.Contains("http")) return url;
     return "http://"+url;

[/code] 

Now the above function is a slim function, not only it is simple to test the function but also it helps to simplify the GetSource() too. The second part we can refactor is, the dependency on the WebRequest and WebResponse. We can write a function that returns a StreamReader and replace it in GetSource(), let's do this

[code:c#]

public StreamReader GetStreamReaderFromWeb(string url)
{
     url = CheckHttpPrefix(url);
     if(url==String.Empty) return null;
     StreamReader readStream = null;
     try
     {
         WebRequest request = WebRequest.CreateUrl();
         WebResponse response = request.GetResponse();
         Stream responseStream = response.GetResponse.Stream();
         Encoding utf8Encode = Encoding.GetEncoding("utf-8");
         readStream = new StreamReader(responseStream,utf8Encode);
      }
      catch
     {
         readStream = null;
     }
     return readStream;     

[/code] 

Now we have a function that checks for http prefix, another function that returns StreamReader. As we return StreamReader from the function, now it should be easy to mock out this function cause any function that returns a streamreader could replace it. Let's recap out GetSource() function changing it so that it takes a streamreader parameter. By doing this, we simplify the unit testing. (cause it is now easier to mock out, if you have the streamreader inside the body of the function not as the parameter, it wont be easy to mockout)

[code:c#]

public string GetSource(StreamReader reader)
{
     if(reader ==null) return String.Empty;
     string htmlSource = reader.ReadToEnd();
     return htmlSource;

[/code] 

Have fun 



Tags:

E-mail | Permalink | Trackback | Post RSSRSS comment feed 0 Responses

Jun162008

Security Problems

One of our web application which is developed using classic asp, has security problems all the time and causing problems to us. I compiled a number of things you should do to escape the same problems, let me know what u think

  • Keep security in ur mind starting the design level
  • use stored procedures or at least parameterized sql
  • keep connection string in web.config and encrypt it
  • validate user input including form inputs, querystring inputs etc
  • encode user input
  • dont give out hacker friendly error messages
  • dont reinvent the wheel for user management, account management, use membership provider
  • encrypt or better hash passwords in the database
  • instead of sql authentication try to use windows authentication to connect to sql
  • dont create too many admin accounts; use delefation
  • log failed attempts
  • monitor your application
  • patch your system
  • When u need ssl, use it
  • dont reinvent encryption algorithms use .net framework security algos
  • dont forget software ages like human being; the older it is, the more complains you will get
  • have a backup plan
  • have backups
  • document ur apps, ur security tests
  • read owasp.org
  • read ms security bulletin


Tags:

E-mail | Permalink | Trackback | Post RSSRSS comment feed 0 Responses

Jun142008

Unit Testing IsUsingIFrame function

Yesterday, i showed a function which does a regular expression check on a passed string, and returns true/false. Below is the code again one more time:

[code:c#]

public bool IsUsingIFrame(string htmlSource)
        {
            if (htmlSource == string.Empty)
                return false;
            string strRegExIFrameCheck = "<iframe\\s+.*src\\s*=\\s*[\"']http://www.csusb.edu/banner2007/?[\"']";
            Regex regexIFrameCheck = new Regex(strRegExIFrameCheck);
            return regexIFrameCheck.IsMatch(htmlSource);
        } 

[/code]

So lets unit test this function using ms unit test. Right click on the function name, and then click on create unit tests, click ok again on the next window. Name your project.

Rename the unit test that vs created for you to "CheckWhenThereisNoIframe" , we are going to pass a string that does not have any iframe and test it. below is the simple code for it

[code:c#]

 [TestMethod()]
        public void CheckWhenThereisNoIframe()
        {
            Scraper target = new Scraper(); // TODO: Initialize to an appropriate value
            string htmlSource = "THERE IS NO FRAME TAG HERE"; // TODO: Initialize to an appropriate value
            bool expected = false; // TODO: Initialize to an appropriate value
            bool actual;
            actual = target.IsUsingIFrame(htmlSource);
            Assert.AreEqual(expected, actual);
        }

[/code]

We should unit test the conditions where string is empty, when iframe is there with a wrong pattern, when there is iframe with right pattern, below is my code

[code:c#]

[TestMethod()]
        public void CheckWhenThereisNoIframe()
        {
            Scraper target = new Scraper(); // TODO: Initialize to an appropriate value
            string htmlSource = "THERE IS NO FRAME TAG HERE"; // TODO: Initialize to an appropriate value
            bool expected = false; // TODO: Initialize to an appropriate value
            bool actual;
            actual = target.IsUsingIFrame(htmlSource);
            Assert.AreEqual(expected, actual);
        }
        [TestMethod()]
        public void CheckWhenStringIsEmpty()
        {
            Scraper target = new Scraper(); // TODO: Initialize to an appropriate value
            string htmlSource = string.Empty; // TODO: Initialize to an appropriate value
            bool expected = false; // TODO: Initialize to an appropriate value
            bool actual;
            actual = target.IsUsingIFrame(htmlSource);
            Assert.AreEqual(expected, actual);
        }

        [TestMethod()]
        public void CheckWhenIframeExists()
        {
            Scraper target = new Scraper(); // TODO: Initialize to an appropriate value
            string htmlSource = "<iframe src='http://www.csusb.edu/banner2007/'/>"; // TODO: Initialize to an appropriate value
            bool expected = true; // TODO: Initialize to an appropriate value
            bool actual;
            actual = target.IsUsingIFrame(htmlSource);
            Assert.AreEqual(expected, actual);
        }

        [TestMethod()]
        public void CheckWhenThereisIframeIncorrectPattern()
        {
            Scraper target = new Scraper(); // TODO: Initialize to an appropriate value
            string htmlSource = "<iframe src='www.google.com"; // TODO: Initialize to an appropriate value
            bool expected = false; // TODO: Initialize to an appropriate value
            bool actual;
            actual = target.IsUsingIFrame(htmlSource);
            Assert.AreEqual(expected, actual);
        }

 

tomorrow i will try to mock the webcreate method .

[/code]



Tags:

E-mail | Permalink | Trackback | Post RSSRSS comment feed 0 Responses

Jun132008

Scraping ?

In our campus environment, we do have a banner that we strongly encourage other sites on campus to add the banner inside an iframe. I had to write an application which gets the urls from the database, and check if the site is using or banner or log, here is what we expect to see in the sites:

&lt;iframe src ="http://www.csusb.edu/banner2007" width="100%" height="90" frameborder="0" 
    scrolling="No" title="CSUSB main navigation"&gt;

 so i need a regular expression that checks the above signature, below is the function i wrote:

[code:c#]

   public bool IsUsingIFrame(string htmlSource)
    {
        if (htmlSource == string.Empty)
            return false;
        string strRegExIFrameCheck = "<iframe\\s+.*src\\s*=\\s*[\"']http://www.csusb.edu/banner2007/?[\"']";
        Regex regexIFrameCheck = new Regex(strRegExIFrameCheck);
        return regexIFrameCheck.IsMatch(htmlSource);
    }

[/code]

It is a simple function which uses regular expressions to see if there is any pattern of iframe with the specified source. now the second part of the problem is connecting to the sites, to achieve this i use webrequest class as below

[code:c#]

public string GetSource(string Url)
    {
        if (Url == String.Empty)
        {
            this.htmlSource = String.Empty;
            return String.Empty;
        }

        string htmlSource = String.Empty;

        if (!(Url.Contains("http")))
            Url = "http://" + Url;

        try
        {

            //create a web request
            WebRequest request = WebRequest.Create(Url);
            WebResponse response = request.GetResponse();
            Stream responseStream = response.GetResponseStream();
            //himm encoding ? do we need this if everything is english? dunnp
            Encoding utf8Encode = Encoding.GetEncoding("utf-8");
            StreamReader readStream = new StreamReader(responseStream, utf8Encode);
            htmlSource = readStream.ReadToEnd();
        }
        catch
        {
            htmlSource = String.Empty;
        }
       
        this.htmlSource = htmlSource.ToLower();
        return htmlSource;
    }

[/code]

tomorrow i ll try to write a unit test agains this :)

have fun



Tags: ,

E-mail | Permalink | Trackback | Post RSSRSS comment feed 0 Responses