VirtualBox cluster and Hadoop

This is to setup a VirtualBox cluster on Mac and then test Hadoop.

VirtualBox download
(How To Enlarge a Virtual Machine’s Disk in VirtualBox or VMware)
Ubuntu download (use server version)

Running Hadoop on Ubuntu Linux (Single-Node Cluster)
Setup Virtual Hadoop Cluster under Ubuntu with VirtualBox

iPhone development

Bought a book on Beginning iOS Programming, by Nick Harris. This is a very good book imo. It directs you through the entire process of building a Bands application, which stores your favorite music bands, and allows you to search online about band information, find nearby stores in map, and listen to preview samples of the band's tracks on iTunes, it also allows you to share the band's information with other people by email or messages. So by following the book, you instantly build a full functional iPhone application with rather complex functionalities. - I got through the book in 3 weeks, with this finished Bands application on my iPhone.

But when started trying the example project, I found it's using Xcode 5 for iOS 7. The book didn't even mention which version of Xcode it's using.

I'm still running Xcode 4.1 on OS X 10.7.5, a.k.a OS X Lion. 10.8 is Mountain Lion, and 10.9 is Mavericks which was released on 10/22/2013. Mavericks is the first free upgrade of OS X. However there are quite some security media coverage on it.

Without a time machine to backup Lion, should I give up upgrading to Mavericks? After reading many of the terrible feedback on line I don't want to upgrade for now.

Why these major software upgrade prematurely only to replace stable old version and ruin their reputation and cause damage to their users? There is a point that when cross it it's overdone.

Maybe I'll stop working on iOS programming now and just concentrate on Android development.

Or let me try virtual machine. VirtualBox is free on Mac. Try to install VirtualBox and see if I can install OS X 10.8 Mountain Lion  and then upgrade to Maverick. Note that Mountain Lion needs to be 10.8.4 or above to run Xcode, so if it's 10.8.3 then you need to find the upgrade. Yikes! It works!  So now I can run Xcode 5.1 in Mountain Lion from VirtualBox on my Lion iMac!

About sharing files between host and guest, it can use the shared folder option in VirtualBox guest OS setting. This needs to install GuestExtension on the guest machine. The GuestExtension is often included in the OS X installation kit, or can be downloaded separately from http://download.virtualbox.org/virtualbox/. However for Mac guest, there is no way to use shared folder this way, because GuestExtension is not available for Mac. So the alternative is to use file sharing on a network. For this, the guest machine should set network as "Bridged Network" instead of "NAT". Then the host and guest can share files on the local network. The file transfer speed can be as fast as hundreds of MB per second.

To remote desktop to PC from a mac, one needs to install Microsoft Remote Desktop Connection Client for Mac.

To install Mountain Lion on VMWare on PC seems hard. But use VirtualBox is fine.

Android App Development

7/12/2014

Open Android emulator and install apk in command line on a mac:

- Start avd manager:
In /Developer/Applications/Android/android-sdk-macosx/tools, type
android avd

- To install an apk
with the emulator running, go into ../platform-tools, type:
./adb install ~/Downloads/myapp.apk


7/3/2014

Now put the Birthday Calculator project on repository, and created the first release.

The App is built with the setting of Android SDK min version 11 and target version 19. In other words, to install this App, you should have Android 3.0 (Honeycomb, see Wiki: Android version history) or above to be safely compatible.


6/26/2014

- After 3 weeks, at the same time I read Nick Harris's Beginning iOS Programming and finished a Bands App on my iPhone. In comparison the "Android Essentials" book did not impress me. It is hard to read and too crude to make much sense. Anyway, it was published back in 2008, so let's don't push too much expectation on it. The only thing "good" is that it's a thin book. Actually, there are tutorials on line. Once got through several prototype examples, one will understand the basic building blocks and underlying framework, then pick a better book becomes easy.

6/13/2014

- Was able to run example projects, and create empty Android project.
- downloaded book code from http://www.apress.com/9781430210641
  for book Android Essentials, by Chris Haseman. QA76.76 A65 J37 2008
- Android emulator runs too slow. Follow this to make it faster:
  http://stackoverflow.com/questions/1554099/why-is-the-android-emulator-so-slow
  - save this html page as "performance - Why is the Android emulator so slow.pdf"
  Basically:
  Use 2000 MiB for SD Card. Enable Snapshot. Use 1024 for device ram size.
  In AVD manager, Start.. -> check Launch from snapshot and Save to snapshot
  This seems faster but not good enough. So try the next: HAMX acceleration.
- My CPU: Intel Xeon CPU W3530. Support vt, from
  http://ark.intel.com/Products/VirtualizationTechnology
  Tried the one using HAMX acceleration. It's much faster now!
  -> The HAEM.exe is at D:\Android\android-sdk\extras\intel\Hardware_Accelerated_Execution_Manager
- Also found that, once the emulator is on, open new Android app
  does not need to restart the emulator.
- Another online tutorial:
  https://developer.android.com/training/basics/firstapp/index.html
  - this provide info on doing this from command line, v.s. using IDE such as eclipse.
- Same installation procedures work on both Windows and Mac.
  Environment set up on both Windows and Mac.
    

6/12/2014

- Install Eclipse from eclipse.org/downloads/. Use JEE version.
  - 32-bit version does not work.
  - 64-bit version works.
  - tried to create the Hello World application and works.

- Install Android SDK
  - from https://developer.android.com/sdk/index.html
  - run it. Install to D:\Android\android-sdk

- Install Eclipse Plug-In of Android. ADT: Android Developer Tools.
  - From https://developer.android.com/sdk/index.html
    Choose: USE AN EXISTING IDE. It is on page:
    https://developer.android.com/sdk/installing/index.html


6/11/2014

- Borrowed book "Android Essentials" by Chris Haseman from library. It's on web at here.


References:
[1] Develop Apps | Android Developers Homepage
[2] Android ActionBar完全解析，使用官方推荐的最佳导航栏

http://www.jiathis.com/

http://www.jiathis.com/

提供分享到微博、QQ、人人等代码...通过访客不断的分享行为，提升网站的优质外链、增加社会化流量、带来更多的用户！

Bulk Insert with error Violation of PRIMARY KEY

In MSSQL 2008, when doing a bulk insert, it sometimes gets into the error of "Violation of PRIMARY KEY".

It should report the violation key value: "The duplicate key value is (...)". Then one can remove the duplicate and insert again.

However, sometimes this is not reported, due to unknown reason; or sometimes one just wants to continue the insert without interruption.

This can be achieved by adding the "WITH (IGNORE_DUP_KEY = ON)" option when creating the table, at the cost of a much more complex execution plan.

An alternative is to add a temporary unique key on the same column as the primary key, adding the "WITH (IGNORE_DUP_KEY = ON)" option on this temporary key and do the insert. After insert is done, remove the temporary key.

See A creative use of IGNORE_DUP_KEY for the details, and more discussions.

NOTE: for bulk insert, if it fails (like due to violation of PK constraint), then none record is inserted at all.

Bulk insert syntax:

BULK INSERT Test1
FROM 'C:\\temp\test1.txt'
WITH
(
   FIELDTERMINATOR = ',',
   ROWTERMINATOR = '\n'
)

Add a unique index with IGNORE_DUP_KEY on:

ALTER TABLE dbo.Test1
ADD CONSTRAINT UQ_idk
UNIQUE NONCLUSTERED (objID)
WITH (IGNORE_DUP_KEY = ON, ONLINE = ON);

Remove the index:

ALTER TABLE dbo.Test1
DROP CONSTRAINT UQ_idk

Protect unix/linux directory using .htaccess and .htpasswd

This is a note on protecting unix/linux directory using .htaccess and .htpasswd.
The article below is copied from [1].

References:
[1] HTPassword Tutorial
[2] Password Protection with .htaccess & .htpasswd
[3] Apache HTTP Server Tutorial: .htaccess files

Password Tutorial

If you would like to have a set of web pages that are protected, requiring a username/password to gain access, this tutorial will show you how to set it up. This is geared towards the Unix Apache httpd servers used on holly, lamar, and www.colostate.edu. If you are using another web server, you'll need to check that server's documentation to see how to do this.

Steps to Password-protect a Directory

First, create a subdirectory in your web area. For the sake of this tutorial, I have created the "protect" directory. Set the permissions on the directory so that the server has read/execute. I do this by using the local command chgrp-www to set the group to the www group. This is the group that the server runs under at Colorado State University for the lamar, holly and www servers. I have used the -sd flag which sets "set group id" for a directory. This will then force any files you create within the protect directory to the www group, so if you ftp files to this directory they will be automatically readable by the server but not by any other user on the system. I then cd into the protect directory.

cd ~ric/public_html
mkdir protect
chmod g+r,g+x,o-r,o-x protect
chgrp-www -sd protect
cd protect

Next you must create a .htaccess file inside the directory you want protected. You can use either the vi or pico editors on the supported systems mentioned above or ftp the file to this directory. If you are new to unix or know little about vi then I suggest you use the pico editor or ftp the .htaccess file. The command to edit with pico is "pico .htaccess". The .htaccess file should contain the following lines. The items in bold are things you will want to change depending on the location of the AuthUserFile and content of AuthName.

AuthUserFile /z/ric/secret/.htpasswd
AuthGroupFile /dev/null
AuthName "Ric's protected files"
AuthType Basic

<Limit GET>
require valid-user
</Limit>

The AuthName is what the user will see when they're prompted for a password - something to the effect of "Enter the username for Ric's Protected files". The AuthUserFile is location of the password file and should be not accessible with a url on the server for security reasons. This is a full unix path and the permissions should be set up like the "protect" directory using the chmod and chgrp-www commands above so the only one that can read this file is the owner and the server. To get the full path of a directory, cd to that directory and enter the command "pwd" to print the working directory path. Now you'll have to set up the password file. You'll need to use the htpasswd program. It is included with the Apache httpd server.
First cd to the directory that contains the password file. In this example the password file is called .htpasswd and is in the directory /z/ric/secret/ as indicated by the AuthUserFile file entry in the .htaccess file. For every username you want to add to the password file, enter the following. (the -c is only required the first time; it indicates that you want to create the .htpasswd file).

   cd
   mkdir secret
   cd secret
   htpasswd -c .htpasswd pumpkin
     [ you're prompted for the password for pumpkin]
     [ if you have other users enter the following. Don't use the -c]
   htpasswd .htpasswd user2
   htpasswd .htpasswd user3

Again, make sure the permissions are set up like the "protect" directory using the chmod and chgrp-www commands above so the only one that can read files in the "secret" directory is the owner and the server. Here is the protected page using the above setup to password protect this page. The username is "pumpkin" and password is "pie".

DOS Batch file to get the number of rows in all the tables and views in a MSSQL database

Store the code below as a bat file, can obtain the number of rows in all the tables and views.--
--

 @echo off

  set db=Northwind

  FOR /F "skip=2" %%G IN ('sqlcmd -E -S localhost -d %db% -Q "set nocount on; set ansi_warnings off; select TABLE_NAME from INFORMATION_SCHEMA.TABLES order by table_name;"') DO (
    REM echo dump table %%G ..
    FOR /F "skip=2" %%X IN ('sqlcmd -E -S localhost -d %db% -Q "set nocount on; set ansi_nulls on; set ansi_warnings on; select count(*) from %%G;"  -s "," -W') DO (
        echo [%%G]: %%X
    )
  )