http://www.jiathis.com/

http://www.jiathis.com/

提供分享到微博、QQ、人人等代码...通过访客不断的分享行为，提升网站的优质外链、增加社会化流量、带来更多的用户！

Some useful tips for web dev

Prevent Session Timeout in C#.NET.

Convert Chinese characters to Unicode
Whois IP
HTML Color Codes
Random.org
http://sqybi.com/blog/

Some easy-to-use jQuery plugins:

- File upload
- Autocomplete
- Watermark
- Light box

Others:

- http://coolshell.cn
- Ajax tools: http://coolshell.cn/articles/9.html The one on map (Drastic Map) is useful
- Online editor: http://www.coderun.com/ide/

High Scalability Blog

The High Scalability blog is on large data web applications. A very good site!

Web RIA technology comparison

Bubblemark animation test: Silverlight (JavaScript and CLR) vs DHTML vs Flash (Flex) vs WPF vs Apollo vs Java (Swing)

Dropdown menu

Drop-Down Menu: 30+ Useful Scripts to Enhance Header Navigation
IF Internet Explorer THEN Do Something Else (A How To…)

SQL Injection Attack

One of the servers is hacked. The symptom is that javascript code were inserted into some database tables. When users visit the site, the javascript code would invoke remote site scripts, display a faked virus scan and report (might be a dynamic gif image), and ask the user to download and install a virus removal software. Once the user installs the software, his computer will be infected.

A script is written to search and clean the database based on signature (characteristic substring) in the javascript code. SQL Query log is added to record all the executed queries. Nothing was found. The problem was finally identified by checking the web server visit log. There are visit requests where SQL commands such as UPDATE used in query string parameter value. It takes advantage of the fact that two consecutive SQL statements can be executed one after the other. This SQL injection attack achieved its goal. Checking the request parameter before executing the SQL can catch such problems. Use of stored procedure and avoid this vulnerability.

A perl script is written to analyze the visit log files and extract these attack requests. The attacked page and attacker's IP is found. Whois service is used to locate where the attacker's IP is from. Patches were made to the attacked page and the site.

Below is the perl script to analyze web server visit log. It extract queries using the 'update' command. Actually it's found that command 'select' is also used in other queries. Can change the script to extract those as well.

#!/usr/bin/perl

#
# @Author: Tom
# @Created on: 9/30/2010
# @Last modified: 9/30/2010
# @Usage example: perl Analyzer.pl < ex100920.log > 100920.txt
# All lines containing the substring "1091+update" are extracted.
#

use strict;

my @fields;
my $fields_ct;
my $line_ct = 0;
my $attack_line_ct = 0;
my @attacked_Pages;
my @attack_IPs;
my $v;
my @vs;
my $i;
my $prefix;

my @ascii_table = (
"NUL",
"SOH",
"STX",
"ETX",
"EOT",
"ENQ",
"ACK",
"BEL",
"BS",
"HT",
"LF",
"VT",
"FF",
"CR",
"SO",
"SI",
"DLE",
"DC1",
"DC2",
"DC3",
"DC4",
"NAK",
"SYN",
"ETB",
"CAN",
"EM",
"SUB",
"ESC",
"FS",
"GS",
"RS",
"US",
" ",
"!",
"\"",
"#",
"\$",
"%",
"&",
"'",
"(",
")",
"*",
"+",
",",
"-",
".",
"/",
"0",
"1",
"2",
"3",
"4",
"5",
"6",
"7",
"8",
"9",
":",
";",
"<",
"=",
">",
"?",
"@",
"A",
"B",
"C",
"D",
"E",
"F",
"G",
"H",
"I",
"J",
"K",
"L",
"M",
"N",
"O",
"P",
"Q",
"R",
"S",
"T",
"U",
"V",
"W",
"X",
"Y",
"Z",
"[",
"\\",
"]",
"^",
"_",
"`",
"a",
"b",
"c",
"d",
"e",
"f",
"g",
"h",
"i",
"j",
"k",
"l",
"m",
"n",
"o",
"p",
"q",
"r",
"s",
"t",
"u",
"v",
"w",
"x",
"y",
"z",
"{",
"|",
"}",
"~",
"DEL",
);

while(<>) {
  $line_ct ++;
  if (/^\s+$/) { next; } # ignore empty line.
  if (/id=1091\+update/) {} else { next; } # ignore non-attack lines.
  $attack_line_ct ++;

  print $line_ct . ": \n" ;
  print $_ . "\n"; 
  print "Fields dump: \n";
  @fields = split(' ', $_);
  $fields_ct = @fields;
  for ($i = 0; $i < $fields_ct; $i ++) {
    print "$i: ";
    $v = $fields[$i];
    if ($i == 6) {
      $v =~ s/\%2B/+/g;
      #print "$v\n";
      @vs = split('\+', $v);
      foreach my $u (@vs) {
        if ($u =~ /^varchar\(8000\)/) {
          print "$u";
        } elsif ($u =~ /(cast\()(char\()((\d)+)\)/) {
          print "cast(" . $ascii_table[$3]; 
        } elsif ($u =~ /(char\()((\d)+)\)/) {
          print $ascii_table[$2]; 
        } else {
          print "$u";
        }
        #print "+";
        print " ";
      }
      print "\n";
    } elsif ($i == 5) {
      if (Page_Exists($v) == 0) {
        push(@attacked_Pages, $v);
      }
      print "$v\n";
    } elsif ($i == 9) {
      if (IP_Exists($v) == 0) {
        push(@attack_IPs, $v);
      }
      print "$v\n";
    } else {
      print "$v\n";
    }
  }
  print "\n";
}

print "$attack_line_ct attack requests found.\n";

my $Page_ct = @attacked_Pages;
print "$Page_ct attacked Page(s) found: \n";
foreach my $p (@attacked_Pages) {
  print "$p\n";
}

my $IP_ct = @attack_IPs;
print "$IP_ct attacking IP(s) found: \n";
foreach my $p (@attack_IPs) {
  print "$p\n";
}


sub Page_Exists() {
  my ($ip) = @_;
  foreach my $p (@attacked_Pages) {
    if ($ip eq $p) { return 1; }
  }
  return 0;
}

sub IP_Exists() {
  my ($ip) = @_;
  foreach my $p (@attack_IPs) {
    if ($ip eq $p) { return 1; }
  }
  return 0;
}

For cleaning the database, an ASP script is written.

'
' Remove injected code from database tables.
' Use signature and full_signature to specify the injected code.
' Use request string "doClean=y" to do cleaning.
' If not use doClean, will only show infected rows.
' Tom 9-24-2010
'
Dim signature, full_signature
signature = "</script>" 
full_signature = "<script src=http://.../...js></script>"

Dim doClean
doClean = false
if request("doClean") <> "" then doClean = true

Response.Write("<p>Use doClean request parameter to do cleaning. ")
Response.Write("doClean = " & doClean & "</p>")
call getTables()

function getTables()
    'Response.Write("test()<br>")
    Dim db, rs, sql, count, tbl, infectedTblCount
    infectedTblCount = 0

    sql = "select table_name as Name from INFORMATION_SCHEMA.Tables where TABLE_TYPE ='BASE TABLE'"
    set db = Connect()
    set rs = ExecuteRS(db, sql)
    
    count = 1
    do while not rs.eof
        tbl = rs("Name")
            Response.Write("Table " & count & ". " & tbl & "<br>")
            if getTableColumns(tbl) > 0 then infectedTblCount = infectedTblCount + 1
            count = count + 1
        rs.MoveNext()
    loop
    
    Response.Write("<p>" & infectedTblCount & " tables are infected.</p>")

    call rs.close()
    set rs = nothing
    call db.close()
    set db = nothing    
end function


function getTableColumns(tbl)
    Dim db, rs, sql, col, str, chk

    sql = "select column_name as Name from INFORMATION_SCHEMA.COLUMNS where TABLE_name ='" & tbl & "'"
    set db = Connect()
    set rs = ExecuteRS(db, sql)
    
    count = 0 ' count of infected columns.
    
    do while not rs.eof
        col = rs("Name")
        chk = checkTblColumn(tbl, col)
        str = str & "<li>" & col & chk & "</li>"
        if len(chk) > 0 then 
            count = count + 1
            if doClean then str = str & cleanTblColumn(tbl, col)
        end if
        rs.MoveNext()
    loop
    str = "<ol>" & str & "</ol>"
    Response.Write(str)
    
    getTableColumns = count

    call rs.close()
    set rs = nothing
    call db.close()
    set db = nothing
end function


function checkTblColumn(tbl, col)
    Dim db, rs, sql, val, str

    sql = "select " & col & " as Name from " & tbl
    set db = Connect()
    set rs = ExecuteRS(db, sql)
    
    count = 0   
    str = ""
    do while not rs.eof
        val = rs("Name")
        if InStr(1, val, signature) > 0 then
            str = str & ("<li>Infected row: " & encodeStr(val) & "</li>")
            count = count + 1
        end if
        rs.MoveNext()
    loop
    if count > 0 then
        str = "<font color='red'>" & count & " rows infected</font>" & str
        str = "<ol>" & str & "</ol>"
        'Response.Write(str)
        checkTblColumn = str
    else
        checkTblColumn = ""
    end if

    call rs.close()
    set rs = nothing
    call db.close()
    set db = nothing
end function


function cleanTblColumn(tbl, col)
    Dim db, rs, sql, val, str

    set db = Connect()
    set rs = Server.CreateObject("ADODB.Recordset")
    rs.Open tbl, db, 1, 2, adCmdTableDirect
    
    count = 0   
    str = ""
    do while not rs.eof
        val = rs(col)
        if InStr(1, val, signature) > 0 then
            rs(col) = replace(val, full_signature, "")
            rs.Update()
            count = count + 1
        end if
        rs.MoveNext()
    loop
    if count > 0 then
        str = "<font color='red'>CLEANED " & count & " rows infected</font>" & str
        cleanTblColumn = str
    else
        cleanTblColumn = ""
    end if

    call rs.close()
    set rs = nothing
    call db.close()
    set db = nothing
end function


function encodeStr(s)
    s = replace(s, "<", "&lt;")
    s = replace(s, ">", "&gt;")
    encodeStr = s
end function

BTW, this is a wiki article on SQL injection. Search on google for SQL injection would bring up much more articles. Good and important to know for web developers, for security concern.

Tips on IMS development

Here are some tips on information management system web application development from my work. Most are platform independent and can be easily applied to different platforms and technologies. They can also be applied to non-web applications, and are actually general design tips.

• Menu. The menu can be dynamically generated from database. This allows flexible configuration and update. If there are multiple roles, each can be configured to draw from the pool of menu items. For this first get a javascript menu template, then create it dynamically. For better performance and void regenerate it each time, such menu can be generated once at creation time, then be stored for later use.
  
  
• Roles and permissions. There can be many roles. The permission can be configured this way: for each item that needs permission control, say we need read/write/delete, then can use a string such as "742172". Here each digit stands for the permission setting of one role. The value of a digit is the combined sum of 4/2/1, which stands for read/write/delete, similar to unix.
  
  
• Workflow. A table can be used for this, which records stage and stage transition settings. Each role may have different permissions at different stages. Such a table can easily store these information.
  
  
• Email. If there are emails to be sent, an admin interface can be provided to configure the subject, body and other fields of emails. For specific information, provide some macros that the administrator can use, which will be replaced by program later.
  
  
• HTML Form. Can create a self-defined framework, instead of using that of such as .NET's control. This allows more flexibility, and can be more efficient since .NET's control may keep more information than necessary. To store the state/parameters, such as when you need to ask for confirmation of user, use hidden HTML control.
  
  
• HTML Form fields. The form fields settings can be managed in the database. So there is no need to go to code for update. Everything can be configured from database. Like label management below, adding a new field can automatically insert itself to a table in database with default settings. Given each form and each form field (if needed) a class name, and control display properties from CSS.
  
  
• Label management. An admin interface can be provided for this. When a tag is put on a page, it can automatically check if it's in database already, if not then insert itself in as a unique tag name. This allows domain experts to change it, and frees programmers from such ad-hoc, trivial but time-consuming and distracting chores.
  
  
• The use of coding. Any systematic categorizing paradigms can be implemented by coding for ease of control and ease of versioning. Say we can use the starting code of 001 to stand for version 1, 002 to stand for version 2, etc. This can apply to many things and is highly useful.
  
  
• Use classes as much as possible, decouple business logic from interface files.
  

JEE

It's been some time since I used J2EE in class. Well it's called JEE nowadays. Today reviewed JEE tools.

Tutorials: 
- Spring Tutorial. (Took about 5 hours to finish, spreaded into 3 days)
- Hibernate Tutorial.
- Maven Tutorial. (Took half an hour to finish)
- Another Maven Tutorial.
- Oracle's JEE 5 tutorial

The environment is:
- OS: Windows XP
- Java: 1.60
- Ant: 1.7 (http://ant.apache.org/manual/index.html)
  [Need to setup environment variables ANT_HOME and Java_HOME]
- Apache Tomcat: 6.0.29 (http://tomcat.apache.org/download-60.cgi#6.0.29)
  Install as a service. (32-bit/64-bit Windows Service Installer). Use port 8080.
  Local site: http://localhost:8080
  Homepage root is: $CATALINA_HOME/webapps/ROOT/index.html
  $CATALINA_HOME is installation site of Tomcat.
- Spring: 2.5.0
  Most recent version is at http://www.springsource.org/download
  But I need version 2.5.0, which can be downloaded from 
  https://olex.openlogic.com/packages/spring
  This contains many other utilities. The most recent version package does not.
- Maven: 2.2.1. Obtained here. [Need to set up M2_HOME]
- Eclipse: Java EE IDE for web developers. (http://www.eclipse.org/downloads/)

A little note:
- Step 5.5 code for JdbcProductDaoTests.java should include the following two imports to function:
  import org.springframework.test.AbstractTransactionalDataSourceSpringContextTests;
  import springapp.domain.Product;
  Here the first imported class is in library spring-test.jar. 
  If eclipse can't find this then manually add it to properties->Java Build Path->Libraries.

Some concepts:
- SSH: Struts, Spring, Hibernate.
- EJB3.0+struts2+iBATIS
- jsp+Servlet+jdbc is the basics for all Java frameworks.
- jQuery: light-weighted javascript framework.
- Dojo, extjs: javascript frameworks. Used with JSF.
- SOA: Service Oriented Application
- Apache lucene: text search engine library written in Java
- Nutch: Open-source web-search software, built on Lucene Java.
- Webwork: Java web-application development framework.
- NoSQL
- Local search. e.g. foursquare

Some thoughts so far:
- A lot of work for simple application in Spring.
- JEE tools apply software engineering concepts, that may be the reason 
  it's said to be suitable for large web application development. On the 
  other hand this makes it heavy-weighted. Setup environment and getting 
  familiar with the tools cost lots of time. On this, .NET, PHP, ASP are
  easier to catch up.
- Eclipse eats big memory (over 200MB). 
- Maven is like Ant.

ASP upload file size upper limit

When use multipart/form-data type to upload file in ASP, there is a problem of file size limit. On windows server 2003, this limit is about 200KB (204800 bytes). This is not enough since that's really a small file size in today's standards.

The solution on windows server 2003 (IIS 6.0) is to set a bigger value on variable AspMaxRequestEntityAllowed, which is in systemroot\System32\Inetsrv\Metabase.xml. On earlier versions of IIS, the value of AspMaxRequestEntityAllowed can be set in the registry.

Now since big file upload can take long, it may need to set ASP ScriptTimeout value bigger than default (90 seconds). Can do this by "<%Server.ScriptTimeout[=NumSeconds]%>" [3], or in IIS6.0 can set this in IIS manager [4].

References:
[1] What is the limit on Form / POST parameters?
[2] Description of the MaxClientRequestBuffer Registry Value
[3] ASP ScriptTimeout Property
[4] Setting the ASP Script Timeout (IIS 6.0)

Architecture of high-throughput, scalable web application

The points here are taken from this link (In Chinese).

A. Some notes from the book Building Scalable Web Sites (ISBN 0596102356, 2006, 352 pages).

1. Scale up a web application
   - vertically scale up: increase setting of single machine (memory, CPU).
   - horizontally scale up: add more machines
2. Redundancy
   - Back up: hot back up (online backup), cold back up (offline backup).
3. Load balancing
   - session/session-less load balancing
   - hardware/software load balancing
4. Cache

B. Article by Ni Haitao.

1. Use static html: convert dynamic content to static html.
2. Separate image/graphics server from application server.
3. Use database cluster instead of single database.
4. Use Cache. E.g.
   Apache - mod_proxy, squid; 
   Linux - memcached; 
   PHP - Pear cache, eaccelerator, Apc, XCache.
5. Use mirror site.
6. Load balancing.
   - hardware 4-layer exchange
   - software 4-layer exchange
   - 7-layer exchange