Clusterware and PERL

A couple weeks ago, I wrote about my new project at work. We are implementing single instance databases protected by Oracle Clusterware. Eventually, the thought is that we will upgrade to Real Application Clusters, but for now we should be able to support the applications with single instance databases.

This week we completed the set up of the cluster after a few technical delays. Our final hurdle seemed to be starting the database using Clusterware. Part of the registration process for a resource in Clusterware is to tell Clusterware what script has the actions available for that resource. Oracle provides action scripts for resources such as listeners, databases, and Enterprise Manager Agents in a couple white papers that need to be copied out and implemented. The action scripts that are provided in the papers are in the form of PERL scripts so they are portable. PERL is one of those things that I have always meant to learn, but never really made the time to pick it up. The database action script was failing when we tried to start the database, but we were able to start the database on both nodes without using Clusterware.

With some help we were able to add some logging to the script and sent some of the variable values to a text file. Using this method we were able to narrow down the issue to the check subroutine in the script. Further analysis narrowed the problem down to the following line of code:

$check_proc = qx(ps -aef | grep ora_pmon_$ORACLE_SID | grep -v grep | awk '{print $8}');

When I cut the line out of the script and ran it from the shell, it returned exactly what was expected. When we logged the output from in the PERL script, it returned the entire line from the process. For some reason, the awk part of the command was not being executed. Finally, I realized that PERL was treating the $8 part of the awk command as a variable, but since there was no variable for 8, the awk was not returning the eighth field as expected. A simple backslash in front of the $ took care of the issue:

$check_proc = qx(ps -aef | grep ora_pmon_$ORACLE_SID | grep -v grep | awk '{print \$8}');

About the same time that I figured this out, the person I had been working with at Oracle informed me that I had been using an older version of the script. They have since replaced the line with a better option:

$check_proc = qx(ps -ae -o cmd | grep -x ora_pmon_${ORACLE_SID});

Obviously this is much cleaner and accomplishes the same thing. So I learned two things from this experience. Number one, always check the dollar signs in a PERL script and number two, make sure you have the latest version of the script!