Thursday, March 6, 2014

DOS batch file to dump all tables/views in a MSSQL database

Store the code below as a bat file: dump_csv.bat.
--

@ECHO OFF
REM This script dumps all tables in the given database in CSV format.
REM Author: X.C. 3/6/2014
CLS

set db=Northwind
set mode=%1

IF NOT "%mode%" == "all" set mode=test

echo -----------------------------------------------------------------
echo This script dumps all tables in the given database in CSV format.
echo Author: X.C. 3/6/2014
echo.
echo Usage: dump_csv.bat [mode]
echo   If mode = all, the entire tables are dumped.
echo   If mode = test, only first row of each table is dumped.
echo -----------------------------------------------------------------
echo.
echo Mode: %mode%
echo Database: %db%
echo.
echo ==Tables to dump==

FOR /F "skip=2" %%G IN ('sqlcmd -E -S localhost -d %db% -Q "set nocount on; set ansi_warnings off; select TABLE_NAME from INFORMATION_SCHEMA.TABLES order by table_name;"') DO Echo %%GG

echo.
choice /m "Do you want to continue "
if errorlevel 2 goto Lexit

echo.

IF "%mode%" == "test" (
  FOR /F "skip=2" %%G IN ('sqlcmd -E -S localhost -d %db% -Q "set nocount on; set ansi_nulls on; set ansi_warnings on; select TABLE_NAME from INFORMATION_SCHEMA.TABLES order by table_name;"') DO (
    echo dump table %%G ..
    sqlcmd -E -S localhost -d %db% -Q "set nocount on; set ansi_warnings off; select top 1 * from %%G;" -o output/%%G.txt -s "," -W
  )
)

IF "%mode%" == "all" (
  FOR /F "skip=2" %%G IN ('sqlcmd -E -S localhost -d %db% -Q "set nocount on; set ansi_nulls on; set ansi_warnings on; select TABLE_NAME from INFORMATION_SCHEMA.TABLES order by table_name;"') DO (
    echo dump table %%G ..
    sqlcmd -E -S localhost -d %db% -Q "set nocount on; set ansi_warnings off; select * from %%G;" -o output/%%G.txt -s "," -W
  )
)

:Lexit
echo.
pause


REM Note 1: If do "set ansi_nulls off; set ansi_warnings off", then view information cannot be retrieved.
REM Note 2: To obtain table values only, use: where TABLE_TYPE = 'BASE TABLE'
REM             To obtain view values only, use: where TABLE_TYPE = 'VIEW'


To dump just one table T_data from a database DB, use:

echo dump table T_data ..
sqlcmd -E -S localhost -d DB -Q "set nocount on; set ansi_warnings off; select * from T_data;" -o output/T_data.txt -s "," -W

To bulk insert data from generated dump files, use the command below. Note the "FIRSTROW" is 1-based and not 0-based. In the dump file, the first row will be column names, the second row is separator line "----", so data starts from the 3rd line.

USE [DB]
GO

BULK INSERT [T_data]
    FROM 'C:\\output\\T_data.txt'
    WITH
    (
        FIRSTROW = 3,
        FIELDTERMINATOR = ',',
        ROWTERMINATOR = '0x0a'
    )
GO


Friday, February 14, 2014

Estimating Data Compression Savings in SQL Server 2008 and 2008 R2

Estimating Data Compression Savings in SQL Server 2008 and 2008 R2

Thursday, February 6, 2014

Python code to log in a site

Below is Python code to log in a site and submit a page by post:

"""
# Script to log in to website and store cookies.
# run as: python web_login.py USERNAME PASSWORD
#
# from: http://martinjc.com/2011/06/09/logging-in-to-websites-with-python/
#
# sources of code include:
#
# http://stackoverflow.com/questions/2954381/python-form-post-using-urllib2-also-question-on-saving-using-cookies
# http://stackoverflow.com/questions/301924/python-urllib-urllib2-httplib-confusion
# http://www.voidspace.org.uk/python/articles/cookielib.shtml
#
# mashed together by Martin Chorley
# modified by HomeTom
#
# Licensed under a Creative Commons Attribution ShareAlike 3.0 Unported License.
# http://creativecommons.org/licenses/by-sa/3.0/
"""

import urllib, urllib2
import cookielib
import sys

class WebLogin(object):

    def __init__(self, username, password):
      
        # url for website we want to log in to
        self.base_url = 'http://baseurl.com'
        # login action we want to post data to
        # could be /login or /account/login or something similar
        self.login_action = '/account/login.php'
        # file for storing cookies
        self.cookie_file = 'login.cookies'

        # user provided username and password
        self.username = username
        self.password = password

        # set up a cookie jar to store cookies
        self.cj = cookielib.MozillaCookieJar(self.cookie_file)

        # set up opener to handle cookies, redirects etc
        self.opener = urllib2.build_opener(
            urllib2.HTTPRedirectHandler(),
            urllib2.HTTPHandler(debuglevel=0),
            urllib2.HTTPSHandler(debuglevel=0),
            urllib2.HTTPCookieProcessor(self.cj)
        )

        # pretend we're a web browser and not a python script
        self.opener.addheaders = [('User-agent',
            ('Mozilla/4.0 (compatible; MSIE 6.0; '
            'Windows NT 5.2; .NET CLR 1.1.4322)'))
        ]

        # open the front page of the website to set and save initial cookies
        response = self.opener.open(self.base_url)
        self.cj.save()

        # try and log in to the site
        response = self.login()
        #print response.read()

        data = urllib.urlencode({
            'fieldName1' : 'fieldValue1',
            'fieldName2' : 'fieldValue2',
            'btnSubmit'  : "submit"
        })

        response = self.opener.open("http://baseurl.com/func.php", data)
        print response.read()

      
    # method to do login
    def login(self):

        # parameters for login action
        # may be different for different websites
        # check html source of website for specifics
        login_data = urllib.urlencode({
            'username' : self.username,
            'password' : self.password,
            'btnLogin' : "submit"
        })

        # construct the url
        login_url = self.base_url + self.login_action
        # then open it
        response = self.opener.open(login_url, login_data)
        # save the cookies and return the response
        self.cj.save()
        return response


if __name__ == "__main__":

    args = sys.argv

    # check for username and password
    if len(args) != 3:
        print "Incorrect number of arguments"
        print "Argument pattern: username password"
        exit(1)

    username = args[1]
    password = args[2]

    # initialise and login to the website
    WebLogin(username, password)

Wednesday, January 15, 2014

Javascript big int operation

Javascript int are actually float. When '&' bit-wise AND is used, the value is converted to integer and operated on. It's 32 bits only, and is signed by default. So for big int longer than this, there is no way to do bit-wise AND.

Here is a solution: first convert the input integer string (a string, not a int) into binary format, then do a bit by bit comparison. Code is below.

function BIGINT_AND(a, b) {
  a = toBinary(a);
  b = toBinary(b);

  for (var i = a.length - 1, j = b.length - 1; i >= 0 && j >= 0; -- i, -- j) {
    if (a[i] == 1 && b[j] == 1) return 1;
  }

  return 0;
}

// convert from decimal format to binary format
function toBinary(decNum){
    return parseInt(decNum,10).toString(2);
}

Monday, January 13, 2014

Java web development review

Just want to review some basic java web development technologies, such as JSP, JSTL and servlets.

Online search on topic "JSP Tutorial" 

Below tutorials are listed in the order of from simple to more complex:

JSP Tutorial - very very basic tutorial
Tutorials Point: Basic JSP Tutorial - a little more detailed
Servlet
 
Building Web Apps in Java: Beginning & Intermediate Servlet & JSP Tutorials
The Java EE 5 tutorial





Blog Archive

Followers