SQL Clustered Index

A primary key is usually a unique clustered index. A primary key cannot be null.

A unique clustered index, if not a primary key, can be null.

A clustered index determines the physical order of rows in a table.

There can be at most one clustered index on a table. Non-clustered index is built upon clustered index, and can be created only when a clustered index already exists on the table.

A unique key is non-clustered by default, unless you specify it to be clustered.

[1] Creating Clustered Indexes
[2] Using Clustered Indexes
[3] CREATE INDEX (Transact-SQL)

MSSQL - Change table design not-allowed error

Sometimes, for example, when you add a column to a table, or when you want to create a primary key field on a table containing millions of rows, you may encounter the dialog message: "The changes you have made require the following tables to be dropped and re-created".

The solution is: From top menu select Tools -> Options -> Designers -> Table and Database Designers, uncheck the box "Prevent saving changes that require table re-creation". When the table is big, you may also need to increase the value of "Transaction Time-out after" textbox.

However, it is recommended not

Query Plan

When a query goes complicated and data involved is huge, query plan reveals the query execution sequence and helps to optimize the query [1].

In MSSQL Server Management Studio, after opening a query window, one can choose "Query -> Display Estimated Execution Plan" to see the estimated query plan, and select "Include Actual Execution Plan" to show actual query plan in the result. [2] is a very good introduction.

[1] Query plan
[2] SQL Server Query Execution Plan Analysis

MacBook Pro with Mac OS X Lion 10.7

I got a MacBook Pro 13" for the Thanksgiving sales Friday. It's $101 off the regular price -- $1098, plus tax it's $1150. I didn't get an Air because an Air lacks wired ethernet jacket and CD/DVD drive. I'll have to buy extra accessories. Air has 128GB super fast SSD hard drive. Pro has ATA drive but it's 500GB. For 13.3" size, Pro is 2.04kg, Air is 1.35kg, which does not create a big difference for me since I don't carry laptop around often.

The current OS is called Lion, it is called Mac OS X Lion 10.7.

So far the feeling is: the 13.3" screen is indeed a little bit small, can't easily display the different windows without interfering each other. Besides a bigger screen, I also prefer a mouse instead of the touchpad. I may buy these separately later, but now let me just setup the environment first.

So now I'm installing softwares to make it a development machine. It already has php, perl, python, ruby, vi, emacs, svn, git, cvs, lex/yacc. I need to have apache, mysql, postgresql, c/c++, java, rails, gcc, make, Xcode, so that I can develop LAMP web applications, iOS applications, and c/c++/java applications.

First I need to enable root account, since many operations will need root permission. See How to Enable the "root" Account on Mac OS X. I used the "2.2 Directory Utility - (Mac OS X 10.4 or later)" method.

To setup LAMP, I referred to Setting up PHP, MySQL, and Apache in Mac OSX Leopard, and Installation of MySQL server on Apple Mac OS X 10.7 aka Lion.

To setup pear, I followed How to setup PHP Pear in Mac OS X Lion. To restore a LAMP application I wrote in the past, I need pear::DB, so I use "pear install DB" command to install it, quite easy.

When trying to make php work with mysql, I had a problem "Warning: mysql_connect(): [2002] No such file or directory (trying to connect via unix:///tmp/mysql.sock)". The solution is found in the above link.

From the above link, you may have a socket file as /tmp/mysql.sock or /var/mysql/mysql.sock but php is looking in the wrong place. Instead of moving the socket or editing config files one can just create a symbolic link so mac finds the required socket. In my case I have /tmp/mysql.sock but not /var/mysql/mysql.sock, so I used this:
cd /var
mkdir mysql
cd mysql
ln -s /tmp/mysql.sock mysql.sock

To install Xcode, I visited Downloads for Apple Developers (needs apple ID). Early versions of Xcode are also available here. The most recent version Xcode 4.2.1 was just released on 11/17/2011. Its major new features are support to iOS 5 and iCloud. But I'm a little conservative. After checking some user reviews on Xcode 4.2.1, which seem to complain about some backward incompatibility issues, I decided to go with the last one, Xcode 4.1, which was released in July. After Xcode is installed, gcc/g++/make all work.

When type "java" or "javac", Lion pops up a dialog box on installing java runtime environment. This installation seems to actually install the sdk as well, since now I can also compile java programs.

Installing rails is also by following hints of Lion. I don't know why the first time it didn't work, complaining about the missing of a ruby header file. But a second try works.

So far I don't have postgresql yet. But that's ok since I won't use it in a short time. My development environment is nearly setup after one day.

OS Tools

KDirStat is a linux/unix disk usage statistics viewer and cleanup tool. Its Windows version is WinDirStat and Mac version is Disk Inventory X.

Escape special char

When transferring data, special characters may be used as separators of records, e.g., row or record separators. In that case you need to escape the special characters from the data, and recover them at the data receiving side. Here is PHP code for this purpose:

test("\.");

function test($s) {
  $t = encode($s);
  $o = decode($t);
  p("s = [$s], encode(s) = [$t], decode(encode(s)) = [$o]. " . (($o == $s) ? "passed" : "failed !!!"));
}

//
// Escape row separator "\n" in a string.
// Encoding scheme:
// \ -> \\
// \n -> \.
//
function encode($s) {
  return str_replace(",", "\;", str_replace("\\", "\\\\", $s));
}

//
// Decoding scheme:
// \. -> \n
// \\ -> \
// Note: can't use the following:
//   return str_replace("\\\\", "\\", str_replace("\.", "\n", $s));
// because it fails for the below case:
//   s = "\.". encode(s) = "\\.", decode(encode(s)) = "\n".
//
function decode($s) {
  // Use "===", since "==" treats 0 as false. See http://www.php.net/manual/en/function.strpos.php
  if (strpos($s, "\\") === false) return $s;

  $t = "";
  $len = strlen($s);
  for ($i = 0; $i < $len; $i ++) {
    $c = $s[$i]; 
    if ($c == "\\") {
      $d = $s[$i + 1];
      if ($d == "\\") { $t .= "\\"; $i ++; }
      else if ($d == ".") { $t .= "\n"; $i ++; }
      else { $t .= "(error)"; } // This shouldn't happen.
    }
    else { $t .= $c; }
  }
  return $t;
}

function p($s) {
  print str_replace("\n", "<br>", $s) . "<br>";
}

Machine Learning Online Course

Machine Learning Online Course by Stanford AI lab director Andrew Ng.

iPhone/iPad hover function

The hover event (can be provided by css or javascript) is not supported in iPad/iPhone.

The workaround is use javascript: "onclick='javascript: f1();'". f1() is defined as:

function f1(o) {
  if ( navigator.userAgent.match(/iPhone/i) || 
       navigator.userAgent.match(/iPad/i) ) { 
    // do the hover work. for example:
    var x = document.getElementById('elemId');
    x.style.left = u.parentNode.offsetLeft + 'px'; // needed for positioning.
    x.style.visibility = 'visible';
  }
  return true;
}

Some sources say one can also use ontouchstart() and ontouchend(). But they didn't work in my situation.

MSSQL partitioning and sp_whoisactive

Partitioned Tables and Indexes in SQL Server 2005

Partitioned Table and Index Strategies Using SQL Server 2008

Who is Active

MSSQL does not support remote scale-valued user define function call.

MSSQL 2008 added new functions merge().

Zip file in windows script

'
' http://superuser.com/questions/110991/can-you-zip-a-file-from-command-prompt
' http://www.rondebruin.nl/windowsxpzip.htm
' http://msdn.microsoft.com/en-us/library/windows/desktop/bb787866%28v=vs.85%29.aspx
'
Dim objArgs, InputFolder, ZipFile, objShell, source, ct

Set objArgs = WScript.Arguments 
InputFolder = objArgs(0) 
ZipFile = objArgs(1) 
CreateObject("Scripting.FileSystemObject").CreateTextFile(ZipFile, True).Write "PK" & Chr(5) & Chr(6) & String(18, vbNullChar) 
Set objShell = CreateObject("Shell.Application") 
Set source = objShell.NameSpace(InputFolder).Items 

' 4 - Do not display a progress dialog box.
objShell.NameSpace(ZipFile).CopyHere source, 4 

'Keep script waiting until Compressing is done
ct = 0
Do Until objShell.Namespace(ZipFile).items.Count = objShell.NameSpace(InputFolder).items.count
   WScript.sleep 1000
   ct = ct + 1
Loop

WScript.Echo "Zip used " & ct & " seconds."

Set source = nothing
Set objShell = nothing
WScript.Quit(0)

Some recent progresses in astronomy and physics

All happen in September and October:

- European experiment reveals the possibility of neutrinos running faster than light. See paper and Youtube video.

- 3 physicists won Nobel Prize for research in the accelerating expansion of the university.

- European Space Agency (ESA) selected the close Solar Orbitor Probe (expected 2017) and Euclid Space mission (expected 2019) projects, to investigate into the sun and the dark matters. See Europeans Planning Closest Solar Probe Ever and Euclid Space Mission Selected by ESA.

File system size limit

There is a limit on how many files or folders you can have in a file system. When that limit is reached the program will bomb.

http://en.wikipedia.org/wiki/Ext3
http://stackoverflow.com/questions/466521/how-many-files-in-a-directory-is-too-many

Trends of technology

It would be of interests to watch the trend of technologies. Below are some websites:

http://trends.builtwith.com/ - show deployment technology of websites.
http://news.netcraft.com/ - show deployment technology of websites.
http://www.google.com/trends/ - show search trends
http://www.tiobe.com/ - for programming languages
http://langpop.com/ - for programming languages. (This site is for sale as of November, 2011)

RoR

RoR is yet another framework for web development. Its major principles are: DRY (Don't Repeat Yourself), CoC (Convention over Configuration) and MVC design pattern.

Ruby was design by Japanese computer scientist Yukihiro "Matz" Matsumoto since mid-1990s. RoR started to become hot back in 2006 due to the release of rails (initial release in 2004) by a Finnish programmer David Heinemeier Hansson. It peaked around end of 2008, since then it is declining (see tiobe index report).

Frankly I don't like RoR very much. It gives you a complete/complicated infrastructure from the very beginning, sure. But following development is restricted to this framework. Building the development environment involves Ruby, Rails is a confusing process, especially when you need to incorporate postgreSQL, Git, Github and Heroku. It's an entire self-contained platform. It adds another layer of abstraction, for example, with the use of HAML and SASS.

RoR Tutorial Book
Starting a Rails app with HAML, Rspec, Devise and Web app theme

Personally, I would prefer PHP, or Python. PHP has a huge body of code repository. Setting up development environment for PHP is much easier.

Twitter has been using RoR, but when scaling the size the site became unstable. Twitter thus updated part of its server end to use Scala in 2008-2009. Now in April 2011, they further replaced the RoR front end to Java, see twitter's engineering log. The Netbeans IDE used to support RoR, but dropped it in 2011, see here.

Ruby will continue to have a seat in its corner. Ever ranked within top 10, and still in top 11 right now is quite an achievement. But I don't think it'll ever sweep the floor as claimed years ago.

Petabytes storage

Petabytes on a Budget v2.0: Revealing More Secrets

Two games on folding protein and RNA

Report: Citizen scientists use games to design protein and RNA

Games:
Fold it
eterna

SVN tips

SVN tricks and tips.

Use transaction in stored procedure

When timeout error occurs, the transaction is left open and wait for user handling. But using "SET XACT_ABORT ON" instructs the server to automatically roll back. It is recommended to always set this on. See:
Use Caution with Explicit Transactions in Stored Procedures.

The Acid3 test

wiki: Acid3
The browser compatibility Acid3 Test.

Go implementation

See the eidogo project. It is written by Justin Kramer.

This is based on the eidogo project.

Another open source Go project is GNUGo.

The History of Programming Languages

The History of Programming Languages

The Evolution of a Haskell Programmer

The Evolution of a Haskell Programmer

Interesting. A lot of ways to achieve the same goal.

An attempt to platform independence compromisation

I got a problem.

I have an application that I hope to be able to: 1) visit directly from its hosting server, and 2) visit on facebook (where it is enclosed in an iframe). I want it to look all right in both a computer and an iphone in both cases.

I was using the following to define the application's width:

var D=navigator.userAgent.toLowerCase();
var x=(D.indexOf("android")!=-1)||(D.indexOf("iphone")!=-1);
this.w0 = x ? (window.innerWidth - 20) : 320;

Use of "window.innerWidth - 20" lets it automatically adjust the width for an iphone. However since facebook embeds my application in an iframe, window.innertWidth will be the width of the iframe window. The application looks big and its bottom half is hidden and covered by the iframe. It does not look right.

I was thinking of letting the application determine if it is in an iframe AND in an iphone, and if so pop up a new window pointing to the original hosting server. Then "window.innerWidth - 20" will adjust the width properly. This failed. It failed because when the application uses parent.location.href to get the parent window, it has permission issue when the application and its enclosing iframe window are on different server. What made it worse, iphone does not allow javascript to pop up a new window, obviously also for security reason.

I thus gave up and use the following:

this.w0 = x ? 300 : 320;

I use 300 because my iphone4 (3G) screen width is 320. This way when visiting facebook in iphone, I still can zoom in the screen to use the application, and it still looks all right to me.

This will look small with larger cell phone screen.

The morale is: platform independence is always a problem. You need different versions for your application :(

find command batch processing

Find some folders (say all .svn folders) and remove them all, in one command:
find -iwholename './*.svn' -exec rm -Rf '{}' \;

程序员技术练级攻略

程序员技术练级攻略
Joel Spolsky's blog
Joel Spolsky: Advice for Computer Science College Students

The Interconnected World of Tech Companies

Very interesting. See how new companies emerge as the result of being established by employees from earlier companies: The Interconnected World of Tech Companies.

iPhone/iPad/android touch event

I have been looking for ways of detecting touch screen events on iPhone comparable to mouse events on a regular computer. I found this today and think it's wonderful. My game now can interact with user on iPhone.

On iPhone/iPad/android, can use the follow replacements for mouse events:
mousedown -> touchstart
mousemove -> touchmove
mouseup -> touchend

In mouse/touch events, obtain page location by:

me.vSelectCanvas.addEventListener('mousedown', function(e) {
  if (e.which == 3) { return; } // do nothing for right mouse button.
  var mouse_x = e.pageX - this.offsetLeft,
      mouse_y = e.pageY - this.offsetTop;
   // ...
}, false);

me.vSelectCanvas.addEventListener('touchstart', function(e) {
  if (e.touches.length == 1) {
    var mouse_x = e.touches[0].pageX - this.offsetLeft,
        mouse_y = e.touches[0].pageY - this.offsetTop;
    // ...
  }
}, false);

To detect browser agent type (regular browser or smart phone) and reset window size:

  var D=navigator.userAgent.toLowerCase();
  var x=(D.indexOf("android")!=-1)||(D.indexOf("iphone")!=-1)||(D.indexOf("ipad")!=-1);
  var d=x?window.innerWidth:900;
  var B=x?window.innerHeight:550;

(See code here)

And here's information on How to get iPad/iPhone screen dimensions in javascript.

Quant

金融工程师Quant的简介

IT new trend

Seems it includes social/mobile/cloud computing, and mining on large data.

Quora - a question-answer addition/search web site (About).
浅析Quora的技术架构
Quora使用到的技术
What's interesting to me is long polling (comet) - a server push technology.

今年科技公司IPO表现点评.
Interesting ones to me include LinkedIn, Zillow, Zygna (简介), Facebook.

如何科学的建立自己的个人网站

The Myth of Architecture Corruption

Baidu Web App Contest

Call a C program from php and read output

Use shell_exec() command:

$output = shell_exec('/path/to/your/program'); // $output contains whatever program prints.

This is great. It makes possible writing C code for computation intensive part and pass result back to php.

Gaming Industry Marketing and Management

游戏市场运营管理 - Good market overview.
【Iphone 游戏开发】游戏引擎剖析 - Good technology overview.
游戏外挂设计技术探讨 - Only interested in the technologies involved.

This is a game I wrote for run recently in HTML5+Javascript+jQuery: Six-Piece Chess.

YUI compressor online is a handy tool to use.

wsdl cache, count_big

PHP web service, cache files are stored in /tmp as "wsdl-..." files. After update web service, you need to delete these files to clear the cache so the changes are reflected.

"select count(*) from tbl" would time out and says: "Arithmetic overflow error converting expression to data type int.". This site has it that you should use "select COUNT_BIG(*) from tbl" instead. Also see MSDN at Transact-SQL reference or COUNT_BIG.

Blob Sallad

Very funny and interesting: Blob Sallad
An article on its implementation: Blob Sallad – canvas tag and JavaScript physics simulation experiment

Dropdown menu

Drop-Down Menu: 30+ Useful Scripts to Enhance Header Navigation
IF Internet Explorer THEN Do Something Else (A How To…)

What Every Programmer Should Know About Memory

What Every Programmer Should Know About Memory

Ulrich Drepper
Red Hat, Inc.
November 21, 2007

Create iPhone application on windows

How To Develop iPhone Apps on Windows
iPhone Development On Windows – 7 Options
Develop iphone Apps on Windows

Seems the top methods are:
1) Flash CS 5.5 (not free)
2) Airplay SDK
3) html/css/javascript
4) In C/C++ (DragonFireSDK) or Java.

Cocoachina: http://www.cocoachina.com/

Fairchild semiconductor

Root of silicon valley semiconductor industry.
Fairchild Semiconductor
仙童半导体公司

Document Management System

Some existing document management systems:

- EndNote. One important function is to create bibliography. Commercial ($25 annual license). Has versions for web, windows, mac.
- Zotero. Collect, organize, cite, and share documents. Competitor of EndNote. Open source. Has versions for web, windows, mac and linux.

I'm looking forward to these functions: record file source, record read time, save local copy, classification, allow comment, can search, can generate report, easy to download and wrap all, can share between users. Better to have iPhone version.

jQuery, Charting

http://jquery.com/
flot - Javascript plotting for jQuery : draws on the canvas element (For IE version earlier than 9, needs excanvas JavaScript emulation helper and canvas-text). Purposes are easy use, attractive look, and interactive features.
Google charts - This is actually closed source. The JavaScript is compacted, and loads libraries dynamically at runtime.

用于图片处理的10个超级jQuery插件
Comparison: Javascript框架和jQuery

PayPal – Authorize.net – Google Checkout

For e-commerce website: PayPal – Authorize.net – Google Checkout

The Connection String Reference

The Connection String Reference: http://www.connectionstrings.com/. Very convenient for use.

ADO Connection String Samples

Character Coding Schemes

编码问题，关于gb2312和utf-8
关于字符编码，你所需要知道的

PHP class to do paging

This code can be easily ported to other languages. Note that the database used here is MSSQL, it has a paging function starting from version 2005 [1][2]. In MySQL, this can be achieved using 'Limit'.

<html>
<head><title>Test</title></head>
<body>
<h1>Test Paging</h1>
<?php 
// General variables for navBar.
$paging = new PagingClass( getService('getDataCount', array()), $_REQUEST['pg'] );
  
// Get data according to range.
$data = getService('getData', array('RangeStart' => $paging->getStart(), 'RangeEnd' => $paging->getEnd()));

// Output navBar and data.
print $paging->writeNavBar();
print Service2Table($data);
print $paging->writeNavBar();
?>

  
<?php

//
// @Author: HomeTom
// @Date: 5/26/2011
//
class PagingClass {  
    
    private $pageSize;
    private $pageButtonCount;
    private $totalCount;
    private $pageCount;
    private $currentPage;
    private $BaseUrl;
    
    //
    // Parameters:
    //   $totalCount: Total number of rows/records.
    //   $curPage: Current page index (usually passed as request parameter).
    // Preassumption: No parameter uses the name "pg", which is used for paging.
    //
    public function __construct($totalCount, $curPage) {
        $this->pageSize = 3;           // Default page size.
        $this->pageButtonCount = 4;    // Default number of paging buttons.

        $this->totalCount = $totalCount;
        $this->pageCount = ceil($totalCount / $this->pageSize);
      
        $this->currentPage = $curPage; 
        if ($this->currentPage == "") { $this->currentPage = 1; }
        else if ($this->currentPage < 0) { $this->currentPage = 0; }
        else if ($this->currentPage >= $this->pageCount) { $this->currentPage = $this->pageCount - 1; }
      
        // Base URL used by page links. Page parameter should be at the end. E.g. "index.php?pg="
        $baseUrl = $_SERVER['PHP_SELF'] . "?" . $_SERVER['QUERY_STRING'];
        if (preg_match("#pg=[0-9]*$#", $baseUrl) > 0) { 
          $this->BaseUrl = preg_replace("#pg=[0-9]*$#", "", $baseUrl) . "pg=";
        } else if ( empty($_SERVER['QUERY_STRING']) ) {
          $this->BaseUrl = $baseUrl . "pg=";
        } else {
          $this->BaseUrl = $baseUrl . "&pg="; 
        }
    }

    // Get start and end row/record number in current page.
    public function getStart() { return $this->currentPage * $this->pageSize + 1; }
    public function getEnd() { return (1 + $this->currentPage) * $this->pageSize; }
  
    //
    // Parameters:
    //   $PageCount: Total number of pages.
    //   $CurrentPageIndex: Current page index.
    //   $PageButtonCount: Number of page buttons to show on one page.
    //
    public function writeNavBar() {
        $PageCount = $this->pageCount;
        $CurrentPageIndex = $this->currentPage;
        $PageButtonCount = $this->pageButtonCount;
        $baseUrl = $this->BaseUrl; //"index.php?pg=";
    
        $DEBUG = 0;
        $lblNext = "Next";
        $lblPrev = "Prev";
        $lblFirst = "First";
        $lblLast = "Last";
    
        $s = "";

        if ($DEBUG) {
            print "pagecount: $PageCount, currentPageIndex: $CurrentPageIndex, ";
            print "PageButtonCount: $PageButtonCount<br>";
        }
  
        $startPage = (floor(($CurrentPageIndex)/$PageButtonCount) * $PageButtonCount);
        if ($DEBUG) print "startpage = $startPage<br>";
    
        $tmp = $PageCount - $PageButtonCount;
        if ($tmp > 0 && $tmp < $startPage) { $startPage = $tmp; }
    
        // First.
        if ($CurrentPageIndex == 0) { $s .= $lblFirst . " "; }
        else { $s .= "<a href=\"" . $baseUrl . "0\">" . $lblFirst . "</a> "; }
       
        // Prev.
        if ($CurrentPageIndex == 0) { $s .= $lblPrev . " "; }
        else 
        { 
            $j = $CurrentPageIndex - 1;
            $s .= "<a href=\"" . $baseUrl . $j . "\">" . $lblPrev . "</a> "; 
        }
      
        // ...
        if ($startPage > 0) { $s .= "<a href=\"" . $baseUrl . ($startPage - 1) . "\">...</a> "; }
    
        for ($i = 0; $i < $PageCount; $i ++) {
            if ($i < $startPage || $i >= $startPage + $PageButtonCount) { continue; }
            if ($i == $CurrentPageIndex) { $s .= " " . (1 + $i); }
            else { $s .= " <a href='" . $baseUrl . $i . "'>". (1 + $i) . "</a>"; }
        }
      
        // ...
        if ($startPage + $PageButtonCount <= $PageCount - 1) {
            $j = $PageButtonCount + $startPage;
            $s .= " <a href=\"" . $baseUrl . $j . "\">...</a> ";      
        }
      
        // Next.
        if ($CurrentPageIndex >= $PageCount - 1) { $s .= " " . $lblNext; } 
        else 
        {
            $j = $CurrentPageIndex + 1;
            $s .= " <a href=\"" . $baseUrl . $j . "\">" . $lblNext . "</a>";
        }
  
        // Last.
        if ($CurrentPageIndex >= $PageCount - 1) { $s .= " " . $lblLast; }
        else { $s .= " <a href=\"" . $baseUrl . ($PageCount - 1) . "\">" . $lblLast . "</a>"; }
  
        return $s;
    }  
}

//
// This function retrieves a web service return result.
// http://www.php.net/manual/en/soapclient.soapcall.php
//
// Parameters:
// $func: Name of the web service function.
// $params: Array of parameters used by this function.
//
function getService($func, $params) {
    try {
        $url = "http://localhost/test/TestService.asmx?WSDL";
        $client = new SoapClient($url);
        $result = $client->$func($params);
        //reset($result); // 'reset' sets the array pointer to the start of the array.
        // 'each' gets the current key/value pair into a separate array.
        list($key, $val) = each($result); 
        //print $key . ": " . $val . "<br>";
        return $val;
    } catch (Exception $e) {
        die ("<br><font color='red'>getService() error: " . $e->getMessage() . "</font><br>");
    }
} 


//
// This function converts web service return result into a table.
//
// rows are delimited by '\n', 
// cols are delimited by '\t'.
//
// explode() v.s. split():
// Biggest difference is explode() takes a delimiter to split by, 
// while split() takes a regular expression. explode is faster.
//
function Service2Table($val) {
    $row_delimiter = "\n";
    $col_delimiter = "\t";
    $DEBUG = 0;
  
    $v = ""; 
    $v_row = "";
    $rows = explode($row_delimiter, $val);
    $row_count = count($rows);
    if ($DEBUG) print "<br>row count: " . $row_count . "<br>";
    for ($i = 0; $i < $row_count; $i ++) {
        $s = $rows[$i];
        if ($s != "") {
            if ($DEBUG) print "$i. $s<br>";    
            $cols = explode($col_delimiter, $s);
            $col_count = count($cols);
            $v_row = "";
            for ($j = 0; $j < $col_count; $j ++) {
                $v_row .= "<td>" . $cols[$j] . " </td>";
            }
            $v .= "<tr>$v_row</tr>";
        }
    }
    $v = "<table border=1>$v</table>";
    return $v;
}

?>

</body>
</html>

On server side, the web service function (in C#) looks like this:

[WebMethod]
public string getData(int RangeStart, int RangeEnd) {
    string row_delimiter = "\n";
    string col_delimiter = "\t";

    string s = "";

    string connStr = ConfigurationManager.ConnectionStrings["LocalDB"].ConnectionString;
    SqlConnection conn = new SqlConnection(connStr);
    conn.Open();
    string sql =  @"with TmpDataTable AS 
        (SELECT *, ROW_NUMBER() OVER (ORDER BY JobID) as 'RowNum' FROM DataTable)
        SELECT * FROM TmpDataTable WHERE RowNum between " + RangeStart + " and " + RangeEnd;
    SqlCommand cmd = new SqlCommand(sql, conn);
    SqlDataReader sdr = cmd.ExecuteReader();

    int fieldCount = sdr.VisibleFieldCount;

    // get column names.
    s = sdr.GetName(0);
    for (int i = 1; i < fieldCount; i++)
    {
        s += col_delimiter + formatStr(sdr.GetName(i));
    }
    s += row_delimiter;

    if (sdr.HasRows) {
        // get column values.
        while (sdr.Read()) {
            s += sdr[0];
            for (int i = 1; i < fieldCount; i++) {
                s += col_delimiter + formatStr(sdr[i]);
            }
            s += row_delimiter;
        }
    }
    conn.Close();

    return s.Trim();
}

[1] Pagination in SQL Server
[2] retrieve specific range of rows in a SQL Server table

PHP upload size limit

Default PHP upload size limit is 2MB. That's too small.

A place to change this limit is in php.ini. Change "upload_max_filesize" and "post_max_size" bigger should save it [1][2]. Another place to increase this is in .htaccess [2]. If PHP SOAP client gives "Error Fetching http headers" warning, then you need to increase default_socket_timeout in php.ini to fix it [3].

[1] Howto optimize your PHP installation to handle large file uploads
[2] PHP Increase Upload File Size Limit
[3] PHP SOAP client giving "Error Fetching http headers"

Javascript PC simulator

Amazing for me: use Javascript to simulate a linux box in a browser:
http://bellard.org/jslinux/

Hammer projection

wiki: Hammer projection

More HTML5 and Javascript

- A list of javascript functions.
- HTML5 canvas - the basics
- Layering multiple canvases: good for avoiding over redrawing.
- Using Multiple HTML5 Canvases as Layers
- Layering Multiple Canvas Elements using JavaScript and EaselJS
- Understanding save() and restore() for the Canvas Context
- Canvas motion blur

- HTML/JavaScript - Select list - Add/Remove Options: DOM, Old school.
- innerHTML, innerText, textContent.
- Snippets: Howto Grey-Out The Screen
- jQuery: Grey out background and preview image as popup
- Javascript Create New Div HTML Element Dynamically
- Canvas tutorial - From Mozilla, a very good one.

- 10个让人眼花缭乱的HTML5和JavaScript效果
- 13 个强大的基于 HTML5 的 Web 应用
- 当设计师遭遇HTML5 - Good article
- HTML5 完胜 Flash 的 7 大特性
- HAKIM
- HTML5 Canvas Cheat Sheet
- Dive Into HTML5
- 10k Apart Contest

- Web audio in chrome
- Building better web apps with a new Chrome Beta

- 深入HTML5: HTML5 本地存储( Local Storage )的前世今生

Several HTML5 web app of my favorite:
- Keylight
- Blob
- FlowPower

Obfuscated Programming Contests

Wiki: Obfuscated code

In C:
Wiki: International Obfuscated C Code Contest
The International Obfuscated C Code Contest Winning entries

In Assembly:
世界编程大赛第一名的3D程序 (or see here)
又是一个64k华丽至极的程序

Quine and Programming languages for fun

Quine is a program that prints itself. See wiki page on Quine. Here is a special Quine that prints a series of programs in 11 languages and eventually comes back to itself.

Here is an article BT雷人的程序语言（大全） about strange programming languages (made for fun in most cases). They are really funny. The end of this article gives these references:

- Esoteric_programming_languages
- Top 13 Most Absurd Programming Languages
- Befunge语言和文言文编程
- 疯狂的编程语言——ENGLISH，Chef Shakespeare
- DM’s Esoteric Programming Languages
- 十大另类程序语言（上）
- 十大另类程序语言（下）

HTML 5

Some cool HTML 5 stuff:

CanvasMol
13 Amazing Examples of HTML5 and CSS3
20款绝佳的HTML5应用程序示例
学习HTML5不可错过的12家国外网站
Create a Drawing App with HTML5 Canvas and JavaScript
3D effect with only html and css: CSS 3D Meninas
Solving the Traveling Salesman Problem with Genetic Algorithms and HTML5 Web Workers
HTML5笔记(1-3)
HTML5 differences from HTML4

Web开发人员应当知道的15个开源项目

C# Mapping a network drive

C# can map a network drive by calling OS commands [1] or using native win32 API [2].

The code below is from [1]:

System.Diagnostics.Process.Start("net.exe", "use K: \\Server\URI\\path\\here");
System.Diagnostics.ProcessStartInfo psi = new System.Diagnostics.ProcessStartInfo();
psi.FileName = "C:\\application.exe";
psi.WorkingDirectory = "K:\\working\\directory";
psi.WindowStyle = System.Diagnostics.
ProcessWindowStyle.Maximized;
System.Diagnostics.Process p =
System.Diagnostics.Process.Start(psi);

[1] http://forum.codecall.net/c-programming/1119-c-mapping-network-drive.html
[2] http://bytes.com/topic/c-sharp/answers/812115-mapping-network-drive-c

Nature: The Future of the PhD

Nature: The Future of the PhD [Nature 472 (20 April 2011)]

Set of articles discussing the current (problematic) system of PhD, from the training of PhD to the career prospects, and speculations on ways to fix the problems.

A problem with process pipe communication in C#

In C#, the Process component communicates with a child process via a pipe. If a child process writes enough data to the pipe to fill the buffer, the child will block until the parent reads the data from the pipe. This can cause deadlock if your application is reading all output to standard error and standard output, for example, using the following C# code.

              Process p = new Process("...", "...");
              p.StartInfo.UseShellExecute = false;
              p.StartInfo.RedirectStandardOutput = true;
              p.Start();
              p.WaitForExit();
              string output = p.StandardOutput.ReadToEnd();

In this instance, both the parent and the child processes would be blocked, as the filled pipe prevents the child process from completing, while the parent process is waiting indefinitely for the child process to exit. The pipe buffer, in my code, was 4096 bytes. If the output reaches this limit, it would hang.

This problem can be solved by moving the ReadToEnd() before the WaitForExit(), as follows..

              Process p = new Process("...", "...");
              p.StartInfo.UseShellExecute = false;
              p.StartInfo.RedirectStandardOutput = true;
              p.Start();
              string output = p.StandardOutput.ReadToEnd();
              p.WaitForExit();

References:
[1] http://bytes.com/topic/c-sharp/answers/459800-redirect-standard-output-size-limit
[2] http://www1.cs.columbia.edu/~lok/csharp/refdocs/System.Diagnostics/types/Process.html

Command line utilities

Many GUI functions have command line counterparts. This makes it possible to write scripts to automate a lot of tasks. The ones here are for the windows platform, specifically here we are talking about windows host script (*.vbs).

1) For SVN, slik svn is a command line version.

2) The sqlcmd command enables running SQL script from command line.

E.g., this runs eg.sql from command line.
sqlcmd -S localhost -d database_name -i eg.sql

This lists all databases on localhost:
sqlcmd -S localhost -i list_databases.sql

list_databases.sql:
select distinct db_name(database_id) AS DATABASE_NAME from sys.master_files group by database_id;
GO

So if you want to do something to certain databases in a server, you can use this to get a list of databases, and check which ones are you need, then construct sql command dynamically.

Just enter sqlcmd, you enter the interactive mode.

3) Build a visual studio solution from command line:

"c:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\devenv.com" c:\projects\Project1\Project1.sln /build Debug /out

Or:

C:\Windows\Microsoft.NET\Framework64\v3.5\MSBuild.exe c:\projects\Project1\Project1.sln

Script can invoke other executables. Executables (e.g. windows service) can invoke scripts. Both can also capture each other's output. This makes rich interaction.

Windows service uses socket in usual. It can use windows remoting (TcpClientChannel) as well.

The following installs a windows service:

C:\Projects\wsService1\wsService1\bin\Debug>InstallUtil /LogToConsole=true wsService1.exe

Adding the /u switch will uninstall it.

When install the windows service under a user account, you will be prompted for the account name and password. The name should be [domain_name]\account_name. If it is a local user (i.e., no domain_name), then it should be .\account_name. Not doing this causes failure.

4) The WMIC command.

The WMIC command is a powerful command line tool to get all kinds of system information, from running processes to OS related. See here for examples.

5) FileSystemObject.copyFolder and XCOPY

FileSystemObject.copyFolder fails when some target files are readonly, this is so even when the OverwriteExisting argument is set to true. The solution is to use XCOPY instead. You may want to use XCOPY this way:

start /WAIT /B XCOPY %1 %2 /R /Y /E /H /Q

/WAIT - wait until this finishes.
/B - do not open another dos window.
For the rest, use XCOPY /? for details. See here for more details.

C# script to strip comment

This code extracts comment from a line of string. Feeding a stream of lines to it, it concatenates the comment and non-comment parts and return them. Comment is defined by "/**/" or "//" as in C/C++.

    /// <summary>
    /// Input: line.
    /// Output: 
    ///   rStr - The non-comment part is concatenated to rStr.
    ///   rCmt - The comment part is concatenated to rCmt.
    /// </summary>
    private void stripComment(string line, ref string rStr, ref string rCmt)
    {
        bool debug = false;
        bool comment_block = false, comment_line = false;
        int index, index2;
        string NEWLINE = "\r\n";
        string s = "";
        string cmt = "";

        if (line.Trim() == "") return; // ignore empty line.

        string subline = line;
        while (subline.Length > 0)
        {
            // if not in both comments mode:
            //     if find /*, start comment_block from the point on.
            //     elsif find --, start comment_line from here to end of line.
            //     Since both "/*" and "--" may exist on this line, find the one occur first.
            if (comment_block == false)
            {
                index = subline.IndexOf("/*");
                index2 = subline.IndexOf("--");
                if (index != -1 && index2 != -1)
                {
                    if (index < index2) { comment_block = true; }
                    else { comment_line = true; }
                }
                else if (index != -1) { comment_block = true; }
                else if (index2 != -1) { comment_line = true; }

                if (comment_block)
                {
                    // comment_block start found.
                    // print the substring before "/*".
                    if (debug) Console.WriteLine(subline.Substring(0, index));
                    s += subline.Substring(0, index);
                    // strip the first part.
                    subline = subline.Substring(index);
                    continue;
                }
                else if (comment_line)
                {
                    // comment_line found.
                    cmt += subline.Substring(index2) + NEWLINE; // comment part.
                    // print the substring before "--".
                    subline = subline.Substring(0, index2);
                    if (subline.Trim() != "")
                    {
                        if (debug) Console.WriteLine(subline);
                        s += subline + NEWLINE;
                    }
                    comment_line = false;
                    break;
                }
                else
                {
                    // is a normal line.
                    if (subline.Length > 0)
                    {
                        if (debug) Console.WriteLine(subline);
                        s += subline + NEWLINE;
                        break;
                    }
                }

            }
            else
            {
                // $comment_block == 1. In comment_block, 
                // search for */, if found, ends comment_block.
                // Note that in comment_block mode, "--" has no effect.
                index = subline.IndexOf("*/");
                if (index != -1)
                {
                    cmt += subline.Substring(0, index + 2); // comment part.
                    comment_block = false;
                    if (debug) Console.WriteLine("comment_block end found.");
                    subline = subline.Substring(index + 2);
                    continue;
                }
                else
                {
                    // entire line is in comment_block.
                    cmt += subline; // comment part.
                    if (subline != line)
                    {
                        if (debug) Console.WriteLine(NEWLINE);
                        s += NEWLINE;
                        cmt += NEWLINE; // comment part.
                    }
                    break;
                }
            }

        }

        rStr += s; 
        rCmt += cmt; 
    }

The Chicago Manual of Style

The Chicago Manual of Style.

Two types: 1) notes and bibliography, 2) author-date.

Ten classic data mining algorithms

Ten classic data mining algorithms.

Software Carpentry

Today finished reading the software carpentry. It introduces the basic principles of software engineering to research scientists and engineers. In doing so in a short and effective manner, it claims to introduce 10% of the software engineering that can meet 90% of the needs. This material becomes open licensed since 2005.

PHPUnit, Hudson CI, Doxygen and Silverlilght

PHPUnit is used for writing unit tests in PHP. This is a first tutorial.

Hudson CI is a continuous integration server system. Can be used for automated unit testing.

Doxygen is a documentation system for many programming languages.

Silverlight is Microsoft's RIA solution coming with MPF. See here for tutorial.

MS SQL schema

Obtain metadata of objects of a database can add lots of flexibility. This can be obtained in the following ways:

1) INFORMATION_SCHEMA
e.g.:
select table_name as Name FROM INFORMATION_SCHEMA.Tables where TABLE_TYPE ='VIEW'

2) sysobjects
e.g.:
SELECT name FROM sysobjects WHERE xtype = 'U' -- Tables
SELECT name FROM sysobjects WHERE xtype = 'V' -- Views
SELECT name FROM sysobjects WHERE xtype = 'P' -- Stored Procedures