Ten classic data mining algorithms

Ten classic data mining algorithms.

Software Carpentry

Today finished reading the software carpentry. It introduces the basic principles of software engineering to research scientists and engineers. In doing so in a short and effective manner, it claims to introduce 10% of the software engineering that can meet 90% of the needs. This material becomes open licensed since 2005.

PHPUnit, Hudson CI, Doxygen and Silverlilght

PHPUnit is used for writing unit tests in PHP. This is a first tutorial.

Hudson CI is a continuous integration server system. Can be used for automated unit testing.

Doxygen is a documentation system for many programming languages.

Silverlight is Microsoft's RIA solution coming with MPF. See here for tutorial.

MS SQL schema

Obtain metadata of objects of a database can add lots of flexibility. This can be obtained in the following ways:

1) INFORMATION_SCHEMA
e.g.:
select table_name as Name FROM INFORMATION_SCHEMA.Tables where TABLE_TYPE ='VIEW'

2) sysobjects
e.g.:
SELECT name FROM sysobjects WHERE xtype = 'U' -- Tables
SELECT name FROM sysobjects WHERE xtype = 'V' -- Views
SELECT name FROM sysobjects WHERE xtype = 'P' -- Stored Procedures

A Subversion Cheat Sheet

SVN can use a GUI interface such as TortoiseSVN, or a command line version such as slik SVN.

---

Setup ssh key for svn so no need to enter password each time: HOWTO: set up ssh keys

---

A short usage note for Subversion.

Often used svn commands are:

Create a Repository

UNIX: svnadmin create /path/to/repository
Windows: svnadmin create d:/path_to_repository

Checking Out a Project - svn checkout

UNIX: svn checkout file:///repository_name/project/trunk project
Windows: svn checkout file:///d:/repository_name/project/trunk project
Network: svn checkout http://host_name/svn_dir/repository_name/project/trunk project

Getting a List of Projects - svn list

UNIX: svn list --verbose file:///repository_name/project
Network: svn list --verbose http://host_name/svn_dir/repository_name/project

Reviewing Changes - svn status

UNIX: svn status

Adding New Files and Directories - svn add

UNIX: svn add file_or_dir_name

Deleting Files and Directories - svn delete

UNIX: svn delete file_or_dir_name
Network: svn delete -m "Deleting project dir" http://localhost/svn_dir/repository/project_dir

Committing Changes - svn commit

Network: svn commit -m "Saving recent changes" http://localhost/svn_dir/repository/project_dir

Updating Your Local Files - svn update

Network: svn update

Get revision number - svn info

svn info | grep Rev

Check log for the 10 most recent entries:

svn log --limit 10
or
svn log -l 10

MSSQL extended property

MSSQL extended property is used to store metadata information on database objects (tables, views, columns, indexes etc.).

- Extended Property is NOT case sensitive.
- MSDN: add, drop, update, list extended properties:
  http://msdn.microsoft.com/en-us/library/ms180047.aspx

- list all extended properties:
  SELECT * FROM sys.extended_properties;
- list all extended properties for columns in all tables:
  SELECT major_id, minor_id, t.name AS [Table Name], c.name AS [Column Name], value AS [Extended Property]
  FROM sys.extended_properties AS ep
  INNER JOIN sys.tables AS t ON ep.major_id = t.object_id 
  INNER JOIN sys.columns AS c ON ep.major_id = c.object_id AND ep.minor_id = c.column_id
  WHERE class = 1;
- Show all the E.P. of a specific columns in a specific table:
  SELECT objtype, objname, name, value
  FROM fn_listextendedproperty (NULL, 'schema', 'dbo', 'table', 'T1', 'column', 'id');
  
Add/drop/update/list the Extended Property of a column:
  
- Show an E.P. of a columns in a table:
  SELECT objtype, objname, name, value
  FROM fn_listextendedproperty ('[E.P. name]', 'schema', 'dbo', 'table', 'T1', 'column', 'id');

- Add an E.P. of a column in a table (the IF part is optional check):

  IF NOT Exists (SELECT * FROM fn_listextendedproperty(
    'Summary', 'schema', 'dbo', 'table', 'T1', 'column', 'id'
  ))
  EXEC sp_addextendedproperty 
    @name = 'caption' 
    ,@value = 'Employee ID' 
    ,@level0type = 'schema', @level0name = dbo
    ,@level1type = 'table', @level1name = 'T1'
    ,@level2type = 'column', @level2name = id;
  GO

- Drop an E.P. of a column in a table (the IF part is optional check):

  IF Exists (SELECT * FROM fn_listextendedproperty(
    'Summary', 'schema', 'dbo', 'table', 'T1', 'column', 'id'
  ))
  EXEC sp_dropextendedproperty 
    @name = 'caption' 
    ,@level0type = 'schema', @level0name = dbo
    ,@level1type = 'table', @level1name = 'T1'
    ,@level2type = 'column', @level2name = id;
  GO

- Update an E.P. of a column in a table:
  EXEC sp_updateextendedproperty 
    @name = N'Caption'
    ,@value = 'Employee ID must be unique.'
    ,@level0type = N'Schema', @level0name = dbo
    ,@level1type = N'Table',  @level1name = T1
    ,@level2type = N'Column', @level2name = id;
  GO

Add/drop/update/list the Extended Property of a table:

- The above shows how to add/drop/update/list the E.P. of a column.
  To do the same thing for a table, just ignore the last 2 items for
  level2type/level2name. For the IF part, use NULL for the last two items. E.g.:
  IF Exists (SELECT * FROM fn_listextendedproperty(
    'Summary', 'schema', 'dbo', 'table', 'T1', NULL, NULL
  ))
  Examples are given below.

- Drop an extended property of a table:  
  IF Exists (SELECT * FROM fn_listextendedproperty(
    'Summary', 'schema', 'dbo', 'table', 'T1', null, null
  ))
  EXEC sp_dropextendedproperty 
    @name = 'Summary' 
    ,@level0type = 'schema' ,@level0name = dbo
    ,@level1type = 'table' ,@level1name = 'T1';
  GO
  
- Add an extended property to a table:  
  IF NOT Exists (SELECT * FROM fn_listextendedproperty(
    'Summary', 'schema', 'dbo', 'table', 'T1', null, null
  ))
  EXEC sp_addextendedproperty 
    @name = 'Summary' 
    ,@value = 'T1 table''s summary' 
    ,@level0type = 'schema' ,@level0name = dbo
    ,@level1type = 'table' ,@level1name = 'T1';
  GO  

Add/drop/update/list the Extended Property of a view or view's column:

- Same as table, except that use "view" instead of "table" as level1type.
  
Note:

- Since single quote "'" is used as delimiter, it should escaped
  by "''" if it is used in the value.
- Note that string quoted by "'" is allowed to contain new line character.
  So if a value spans multiple lines, it won't be a problem.

Perl OOP and reference

Perl can do OOP for sure. I reviewed relevant material for recent work. Use "Package" to define a class. What's interesting is the use of "bless" keyword. An object is used in the syntax of a C++ object, say you use "->" to retrieve its member variable or function.

The use of reference facilitates a lot of programming features. In Perl using a back slash "\" in front of a variable (scalar, array or hash) turns it into a reference. This can be passed to subroutines, to be used recursively etc. To cast it back and refer to the object, use $/@{}/%{} for scalar/array/hash respectively.

References:
- Object Oriented Programming in PERL