Hash

June 7, 2011

Database Schema Versioning

Database schema is an important part of your source code and you would like to release it along with your application especially if it is a web application. The other reason to version your database is for continuous integration. Every time a team member makes some change in the code the build server picks it up and rebuilds the whole application again. If that change contains a database update, you would want that to happen on the build server automatically. Here comes your database script runner tool.

I have been looking for tools like that over the internet but could not find an open source script runner. The one which does the job is not open source and the one which is open source does not do the job right. So I decided to write my own.

This is a class library which takes in a connection string, a file format string and the path to the scripts directory. Connection string is off course the address of your database and the file format is an handy feature so you can use any name format for your sql scripts. For instance: a file name “my_%.sql” will read all files of the format my_<sequence number>.sql where sequence number is a 4 digit integer.

Here is the code. I use it as a library and run it at the start up on one of my services but you can create a console application and reference the assembly in it to use it as a standalone tool. I might create that console application later and will share it here if I do.

    public class SchemaUpdater
    {
        private readonly string _connectionString;
        private readonly string _nameformat;
        private readonly string _scriptdir;

        public SchemaUpdater(string connectionstring, string format, string scriptpath)
        {
            _connectionString = connectionstring;
            _nameformat = format;
            _scriptdir = scriptpath;
        }

        public void Update()
        {
            using (var connection = new SqlConnection(_connectionString))
            {
                connection.Open();
                var scriptseparator = new[] {"\nGO"};

                // Make sure we have a schema versions table
                var scriptfile = string.Format("{0}\\{1}", _scriptdir, "versions-table.sql");
                var transaction = connection.BeginTransaction(IsolationLevel.Serializable);
                try
                {
                    Array.ForEach(File.ReadAllText(scriptfile).Split(scriptseparator, StringSplitOptions.RemoveEmptyEntries),
                        (sql => new SqlCommand(sql, connection, transaction).ExecuteNonQuery()));
                    transaction.Commit();
                }
                catch(Exception)
                {
                    transaction.Rollback();
                    throw;
                }

                // Now run the baseline
                scriptfile = string.Format("{0}\\{1}", _scriptdir, "base.sql");
                transaction = connection.BeginTransaction(IsolationLevel.Serializable);
                try
                {
                    Array.ForEach(File.ReadAllText(scriptfile).Split(scriptseparator, StringSplitOptions.RemoveEmptyEntries),
                        (sql => new SqlCommand(sql, connection, transaction).ExecuteNonQuery()));
                    transaction.Commit();
                }
                catch (Exception)
                {
                    transaction.Rollback();
                    throw;
                }

                // Now run all the files
                var command = new SqlCommand("SELECT Version FROM SchemaVersion", connection);
                var version = command.ExecuteScalar();
                command.Dispose();

                var start = version == null ? 0 : Convert.ToInt32(version)+1;

                var filename = _nameformat.Replace("%", start.ToString("0000"));
                scriptfile = string.Format("{0}\\{1}", _scriptdir, filename);

                transaction = connection.BeginTransaction(IsolationLevel.Serializable);

                try
                {
                    while (File.Exists(scriptfile))
                    {
                        Array.ForEach(File.ReadAllText(scriptfile).Split(scriptseparator, StringSplitOptions.RemoveEmptyEntries),
                            (sql => new SqlCommand(sql, connection, transaction).ExecuteNonQuery()));

                        start++;
                        filename = _nameformat.Replace("%", start.ToString("0000"));
                        scriptfile = string.Format("{0}\\{1}", _scriptdir, filename);
                    }

                    new SqlCommand(string.Format("Update SchemaVersion SET Version={0}", start-1), connection, transaction).ExecuteNonQuery();

                    transaction.Commit();
                }
                catch(Exception)
                {
                    transaction.Rollback();
                    throw;
                }
            }
        }
    }

This is a small piece of code but it works for me, also takes care of the GO separator in the sql scripts and runs all commands in a transaction. It uses my scripts “version-tables.sql” which creates a new version tables in the database if it does not exist and “base.sql” which contain any sql statements which you want to run to create a baseline schema.

July 7, 2010

ASP.NET MVC: Binding arrays in model

Filed under: .NET,TechFactor — scarefactor @ 2:09 am
Tags: , , , , ,

One of the great features of ASP.NET MVC is the automatic model binding. This is one of those things which look trivial but in fact are very very important. This reduces the LoC and complexity by miles. For instance, you want to present a number of text inputs to the user and read multiple locations:

 

<input type="text" name="Location1" id="Location1" />
<input type="text" name="Location2" id="Location2" />
<input type="text" name="Location3" id="Location3" />

 

Now this would be tedious in your action:

 

public ActionResult UserData(string location1, string location2, string2)
{
   // do somthing
   return View();
}

 

And what if you don’t know how many locations would there be? That is where MVC shows the magic. Bring it on:

public ActionResult UserData(string[] locations)
{
   // do somthing
   return View();
}

 

 

Hmmm… pretty simple. Now keep adding location inputs in your view and MVC will bind all that to the location parameter. The key here is that the name of your input controls should be location as well with an index like a proper array.

 

<input type="text" name="location[0]" id="location[0]" />
<input type="text" name="location[1]" id="location[1]" />
<input type="text" name="location[2]" id="location[2]" />

 

The magic of model binding does not end here. That is, if you have another array of postcodes along with locations the method signature of your action will look dirty again. The solution for this in MVC is to bind everything in the model object and pass that object over to the action. Now your model object will look like this:

 

public class HomeModel
{
     public IList<string> Locations { get; set; }
     public IList<string> PostCodes { get; set; }
}

 

 

And the action signature will accept only one parameter which is the model.

 

public ActionResult UserData(HomeModel model)

 

Yes, everything is in that object, pretty neat. Make sure your view inherits from System.Web.Mvc.ViewPage<HomeModel> so it knows about its model type.

 

 

 

Share this post :

January 21, 2010

More competition in Smartphone Market

Filed under: LifeFactor,Mobile — scarefactor @ 1:06 pm
Tags: , , , , , , ,

 

With the introduction of Nexus One by Google the competition in the smart phone market is getting tougher. Google had a share of iPhone already with its search engine and maps products being the default in it and now they have their own smart phone as well which is a direct competitor to iPhone.

Now some recent developments which caught the attention of my eye include the news that Apple is in talks to Microsoft to replace Google in its products with Bing. Now this is pro-active! They have anticipated the danger but this is not the best reaction. Check out the following news:

http://www.smh.com.au/technology/biz-tech/bing-could-replace-google-on-iphone-report-20100121-mmn0.html

This looks like just an immediate course of action. Apple’s dependence on other big names to provide services will not last long as Microsoft’s pink phone is also in the pipeline.

http://digitaldaily.allthingsd.com/20100119/microsoft-to-launch-zune-phone-in-2-months/

There is no other search engine at the moment to replace bing in iPhone. Apple’s best course of action is to focus on their product and work to make it the best as they have done in the past. Everyone would love the iPhone which they already are familiar with. But it looks like Apple will have to withdraw from some of its smart phone market share as others jump in!

 


Share this post : del.icio.us it! digg it! Facebook it! live it! reddit! technorati! yahoo!

November 7, 2009

SQL Server clustering vs. Oracle RAC

 

I did some research on SQL Server clustering a while ago to find out what high-availability and fail-over options does SQL Server provide. Below are my findings. In short SQL Server has fail-over but no high-availability as compared to Oracle RAC.

1. SQL Server 2008 Peer-to-Peer Replication

  • Multiple nodes running their own instances
  • Each node has its own copy of data
  • Every node is a publisher and subscriber at the same time
  • Not scalable because of complex architecture
  • Complex to modify schema
  • Conflicts may arise if two nodes update the same row
  • In case of conflict, the topology remains inconsistent until the conflict is resolved
  • Conflict is resolved manually using the method described in
    http://technet.microsoft.com/en-us/library/bb934199.aspx

2. SQL Server 2008 Mirroring

  • One primary server and more than one mirror instances
  • Periodic Log Shipping between primary and secondary servers
  • Failover process is manual
  • A separate ‘witness’ server can be deployed to automate the fail over
  • Secondary servers  do not participate in any transaction and just wait for the failover
  • Equivalent to Oracle standby database technology

3. SQL Server 2008 Failover Clustering

  • Two servers running on a shared storage
  • All data and logs reside on the SAN and is shared
  • One server performs all transactions and the other waits for the failover
  • Microsoft Cluster Server takes care of the fail over
  • Both instances have separate instance names and one cluster instance name
  • Clients connect to the cluster IP address and cluster instance name
  • Failover is transparent but a delay (in minutes) is required to mount the database on the failover instance and start it
  • There is an application blackout during fail over process
  • Reference (http://msdn.microsoft.com/en-us/library/aa479364.aspx)

4. SQL Server 2008 Active/Active Failover Clustering

  • Two instances running on a shared storage
  • Two different SQL Server databases setup on both servers
  • Active/Active Clustering is effectively two different failover clusters
  • Each node in the cluster is running one primary instance and one secondary instance of the other node
  • Both clusters run a synchronized copy of the database
  • Replication is setup between both clusters to keep them synchronized
  • Clients see two different databases available to connect to
  • In case of failure, one server runs both database instances which may cause performance overhead
  • Write cost may increase because of replication and database synchronization
  • Application blackout will only be for the clients connected to the failed instance
  • Peer-to-Peer replication has conflicts (See No. 1)

5. SQL Server 2008 Federated Database

  • Multiple instances running in a network connected to each other
  • Each instance carry part of the database
  • Complete table is formed always using VIEWS and distributed SQL
  • Each instance has a VIEW of the table using UNION ALL between all instances called DPV
  • Complex to scale up and manage
  • Complex to modify the schema because of multiple databases
  • May have HOT-NODE syndrome when one node carry most used data

6. Oracle 11g RAC

  • Multiple Nodes running on a shared storage
  • All nodes are participating
  • Nodes are connected to each other using inter-network
  • All nodes servicing the single database
  • Scalable because of single database
  • Entire cluster fails if SAN fails
  • Higher performance inter-connect required for cache fusion as nodes increase
  • Virtual IP Address is used to connect to all servers
  • In case of failure of one node, clients will connect to other nodes on the same IP address on subsequent requests
  • 30-60 seconds of delay required for failover
  • Application blackout will only be for the clients connected to the failed instance

 


Share this post : digg it! Facebook it! live it! reddit! technorati! yahoo!

June 12, 2009

Deadlock Resolution: Killing Oracle Sessions

Filed under: CodeProject,Oracle,TechFactor — scarefactor @ 1:20 pm
Tags: , , , , ,

 

We usually face this situation where an Oracle session is dead and an UPDATE command freezes forever. Usually it is the fault of the programmer who messes up with his session. In such case of a deadlock, the problem session needs to be killed.

Following is a query (I got from some other blog I don’t remember) we use to find objects which are locked. This is used to make sure that the object which we are trying to update is actually locked by another session:

select
   c.owner,
   c.object_name,
   c.object_type
   b.sid,
   b.serial#,
   b.status,
   b.osuser,
   b.machine
from
   v$locked_object a ,
   v$session b,
   dba_objects c
where
   b.sid = a.session_id
and
   a.object_id = c.object_id;

And this query tells which are the problem sessions:

select l1.sid, ‘ IS BLOCKING ‘, l2.sid
from v$lock l1, v$lock l2
where l1.block =1 and l2.request > 0
and l1.id1=l2.id1
and l1.id2=l2.id2;

So now we have the SID of the problem session and we only need the serial number to kill it.

SELECT serial# FROM v$session WHERE sid=<sid>

And now kill it:

ALTER SYSTEM KILL SESSION ‘<sid>,<session#>’

Easy!

 

 


Share this post : MSDN! Technet! del.icio.us it! digg it! Facebook it! live it! reddit! technorati!

May 25, 2009

Oracle: Drop Database Command

Filed under: Oracle,TechFactor — scarefactor @ 6:05 pm
Tags: , , , ,

 

Oracle introduced a new command in 10g to drop a database. Before that there was a set of procedures to follow in order to get rid of a database which involved deleting all data files from the file system, deleting control files, the parameter files etc. Now after this command everything is deleted by the RDBMS and there is no chance of forgetting any file to delete.

The command is simple, “DROP DATABASE”.

And the procedure to use this command is:

sqlplus / as sysdba

SQL> shutdown immediate
Database closed.
Database dismounted.
ORACLE instance shut down.

SQL> startup mount exclusive restrict
ORACLE instance started.

SQL> exit

rman target /

drop database;

The database has to be started in exclusive restrict mode.

If you are running a RAC you will have to close down all instances and mount the database exclusively on one node only. The parameter file will need to be changed and the parameter “cluster_database” should be set to false. My suggestion is to create another parameter file from spfile and mount the database using that file.

CREATE PFILE=’/home/oracle/anotherpfile.ora’ from SPFILE;

The file can be modified and all RAC/node specific parameters can be removed.

 

Technorati Tags: ,,,,

 


Share this post : Social! del.icio.us it! digg it! Facebook it! live it! reddit! technorati! yahoo!

May 4, 2009

How to detect a lie

Filed under: LifeFactor — scarefactor @ 4:25 pm
Tags: , , , , , , ,

 

Following is the opinion of some experts about how to detect when someone is lying to you. Technically they sound right but I dare to differ. I don’t think these are the 10 commandments of lie detection. Just use your common sense to analyze your particular situation. I personally like the “extra tip” at the end.

 

J.J. Newberry was a trained federal agent, skilled in the art of deception detection. So when a witness to a shooting sat in front of him and tried to tell him that when she heard gunshots she didn’t look, she just ran — he knew she was lying.

How did Newberry reach this conclusion? The answer is by recognizing telltale signs that a person isn’t being honest, like inconsistencies in a story, behavior that’s different from a person’s norm, or too much detail in an explanation.

While using these signs to catch a liar takes extensive training and practice, it’s no longer only for authorities like Newberry. Now, the average person can become adept at identifying dishonesty, and it’s not as hard as you might think. Experts tell WebMD the top 10 ways to let the truth be known.

Tip No. 1: Inconsistencies

"When you want to know if someone is lying, look for inconsistencies in what they are saying," says Newberry, who was a federal agent for 30 years and a police officer for five.

When the woman he was questioning said she ran and hid after hearing gunshots — without looking — Newberry saw the inconsistency immediately.

"There was something that just didn’t fit," says Newberry. "She heard gunshots but she didn’t look? I knew that was inconsistent with how a person would respond to a situation like that."

So when she wasn’t paying attention, he banged on the table. She looked right at him.

"When a person hears a noise, it’s a natural reaction to look toward it," Newberry tells WebMD. "I knew she heard those gunshots, looked in the direction from which they came, saw the shooter, and then ran."

Sure enough, he was right.

"Her story was just illogical," says Newberry. "And that’s what you should look for when you’re talking to someone who isn’t being truthful. Are there inconsistencies that just don’t fit?"

Tip No. 2: Ask the Unexpected

"About 4% of people are accomplished liars and they can do it well," says Newberry. "But because there are no Pinocchio responses to a lie, you have to catch them in it."

Sir Walter Scott put it best: "Oh what a tangled web we weave, when first we practice to deceive!" But how can you a catch a person in his own web of lies?

"Watch them carefully," says Newberry. "And then when they don’t expect it, ask them one question that they are not prepared to answer to trip them up."

Tip No. 3: Gauge Against a Baseline

"One of the most important indicators of dishonesty is changes in behavior," says Maureen O’Sullivan, PhD, a professor of psychology at the University of San Francisco. "You want to pay attention to someone who is generally anxious, but now looks calm. Or, someone who is generally calm but now looks anxious."

The trick, explains O’Sullivan, is to gauge their behavior against a baseline. Is a person’s behavior falling away from how they would normally act? If it is, that could mean that something is up.

Tip No. 4: Look for Insincere Emotions

"Most people can’t fake smile," says O’Sullivan. "The timing will be wrong, it will be held too long, or it will be blended with other things. Maybe it will be a combination of an angry face with a smile; you can tell because their lips are smaller and less full than in a sincere smile."

These fake emotions are a good indicator that something has gone afoul.

Tip No. 5: Pay Attention to Gut Reactions

"People say, ‘Oh, it was a gut reaction or women’s intuition,’ but what I think they are picking up on are the deviations of true emotions," O’Sullivan tells WebMD.

While an average person might not know what it is he’s seeing when he thinks someone isn’t being honest and attribute his suspicion to instinct, a scientist would be able to pinpoint it exactly — which leads us to tip no. 6.

Tip No. 6: Watch for Microexpressions

When Joe Schmo has a gut feeling, Paul Ekman, a renowned expert in lie detection, sees microexpressions.

"A microexpression is a very brief expression, usually about a 25th of a second, that is always a concealed emotion," says Ekman, PhD, professor emeritus of psychology at the University of California Medical School in San Francisco.

So when a person is acting happy, but in actuality is really upset about something, for instance, his true emotion will be revealed in a subconscious flash of anger on his face. Whether the concealed emotion is fear, anger, happiness, or jealousy, that feeling will appear on the face in the blink of an eye. The trick is to see it.

"Almost everyone — 99% of those we’ve tested in about 10,000 people — won’t see them," says Ekman. "But it can be taught."

In fact, in less than an hour, the average person can learn to see microexpressions.

Tip No. 7: Look for Contradictions

"The general rule is anything that a person does with their voice or their gesture that doesn’t fit the words they are saying can indicate a lie," says Ekman. "For example, this is going to sound amazing, but it is true. Sometimes when people are lying and saying, ‘Yes, she’s the one that took the money,’ they will without knowing it make a slight head shake ‘no.’ That’s a gesture and it completely contradicts what they’re saying in words."

These contradictions, explains Ekman, can be between the voice and the words, the gesture and the voice, the gesture and the words, or the face and the words.

"It’s some aspect of demeanor that is contradicting another aspect," Ekman tells WebMD.

Tip No. 8: A Sense of Unease

"When someone isn’t making eye contact and that’s against how they normally act, it can mean they’re not being honest," says Jenn Berman, PhD, a psychologist in private practice. "They look away, they’re sweating, they look uneasy … anything that isn’t normal and indicates anxiety."

Tip No. 9: Too Much Detail

"When you say to someone, ‘Oh, where were you?’ and they say, ‘I went to the store and I needed to get eggs and milk and sugar and I almost hit a dog so I had to go slow,’ and on and on, they’re giving you too much detail," says Berman.

Too much detail could mean they’ve put a lot of thought into how they’re going to get out of a situation and they’ve crafted a complicated lie as a solution.

Tip No. 10: Don’t Ignore the Truth

"It’s more important to recognize when someone is telling the truth than telling a lie because people can look like they’re lying but be telling truth," says Newberry.

While it sounds confusing, finding the truth buried under a lie can sometimes help find the answer to an important question: Why is a person lying?

These 10 truth tips, experts agree, all help detect deception. What they don’t do is tell you why a person is lying and what the lie means.

"Microexpressions don’t tell you the reason," says Ekman. "They just tell you what the concealed emotion is and that there is an emotion being concealed."

When you think someone is lying, you have to either know the person well enough to understand why he or she might lie, or be a people expert.

"You can see a microexpression, but you have to have more social-emotional intelligence on people to use it accurately," says O’Sullivan. "You have to be a good judge of people to understand what it means."

Extra Tip: Be Trusting

"In general we have a choice about which stance we take in life," says Ekman. "If we take a suspicious stance life is not going to be too pleasant, but we won’t get misled very often. If we take a trusting stance, life is going to be a lot more pleasant but sometimes we are going to be taken in. As a parent or a friend, you’re much better off being trusting rather than looking for lies all the time."

By Heather Hatfield / WebMD Feature

 

 


Share this post : digg it! Facebook it! live it! reddit! technorati! yahoo!

April 27, 2009

Oracle Parameter File: Order of lookup

 

Oracle has two type of parameter files for initial parameter configurations: PFILE and SPFILE. The parameter file and the shared parameter file. They are used to specify parameter values at the database startup. The SPFILE is used in a RAC environment and PFILE is used in a stand alone database environment.

The other day I had a problem starting up my RAC and later on it was discovered that the problem was because of these startup files. The recommended way of using an SPFILE in a RAC environment is to specify the (shared) location of your SPFILE in the PFILE. All nodes have their own PFILEs which point to the same SPFILE which is located on a shared storage. I had set it up all right but it was not working. And this was because Oracle searches the configuration files in the specified directory in the following order:

  • spfileSID.ora
  • spfile.ora
  • initSID.ora
  • init.ora

So I had my initSID.ora setup but was not being used because I already had one spfileSID.ora in the same location which always got preference.

 

 


Share this post : digg it! Facebook it! live it! reddit! technorati! yahoo!

April 20, 2009

Oracle to buy Sun Microsystems for $7.4 bn

Filed under: LifeFactor,TechFactor — scarefactor @ 9:58 pm
Tags: , , , , , , ,

 

In a news today by CNN, Oracle confirmed it will buy Sun Microsystems for $7.4 bn. This also strengthens the belief that Sun refused IBM because it had a better offer.

Acquisition of Sun will be fabulous for Oracle because it can now enter in the servers market and that with a BANG! Sun’s powerful hardware and trusted Operation System ‘Solaris’ will be a huge plus for Oracle but the future of two most widely used open source relational databases remains uncertain. Sun has bought MySQL a couple of years ago and was supporting PostgreSQL as well. What is the use of MySQL for Oracle? And support for PostgreSQL will most probably be dropped.

Sun was a better buy for IBM anyway.

 

 


Share this post : digg it! Facebook it! live it! reddit! technorati! yahoo!

April 14, 2009

Skype Founders want it back

Filed under: LifeFactor,TechFactor — scarefactor @ 12:57 am
Tags: , , , , , , , ,

 

http://www.techcrunch.com/2009/04/10/report-founders-want-to-buy-skype-from-ebay/

 

As the New York Times reported, Skype’s founders, Niklas Zennstrom and Janus Friis, are in talks with several private equity firms and are amassing their own financial resources to make a bid for the internet phone business. eBay bought Skype from Zennstrom and Friis for around $3.1 billion in 2005. We reported last year that eBay would be willing to sell Skype if the company couldn’t support eBay’s core ecommerce business.

eBay has been having trouble finding ways of using Skype across its other products. eBay removed Skype co-founder and CEO Niklas Zennstrom in October 2007, reportedly due to frustration at the financial performance of Skype. Ebay also negotiated down the huge earnout due to Skype stockholders and took a $936 million one-time loss around the transaction.

As we wrote last spring, a sale was projected to be likely late last year or in the first half of this year. Of course, with the economy in such poor condition, the sale was probably put off momentarily. There was something brewing between Google and Skype last spring, but nothing came of it. Google recently launched its own voice product, Google Voice (formerly Grand Central).

Skype recently launched a nifty iPhone app, which was downloaded 1 million times in the first two days of its release. Skype recently made a move to be a player in the enterprise space, but it wasn’t clear how much of a money-maker the new service would be.

There’s no doubt that Skype brings a lot to the table but eBay was probably just not the right buyer. Skype’s scalable technology and a proven platform in the VOIP, VOIP2POTS and P2P Video is impressive to say the least. The service currently has more than 405 million registered users.

Following their respective departures from Skype, Zennstrom and Friis formed VC firm Atomico and founded online video/TV site Joost.

 

 


Share this post : digg it! Facebook it! live it! reddit! technorati! yahoo!

Next Page »

Theme: Rubric. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.