Latest Entries »

Troubles installing Oracle one-off patches

I had never installed one-off patches before and last week I got my first opportunity. I turned out to be a nightmare!! I could not get it to work. Oracle support could not figure out my problem!! At the end of the day it turned out to be 2 very simple things. One of them Oracle support had never encountered!

Errors:
There is no $ORACLE_HOME to apply the patch to.

and
Same error: opendir(/opt/product/IAS/10.1.2.0.2/reports/../../../../..): Permission denied at /opt/product/IAS/10.1.2.0.2/reports/OPatch/perl_modules/Command.pm line 8384

Main problem opatch lsinventory could not accept the ORACLE_HOME for the reports server.

First solution. We found /etc/oraInst.loc pointing to a different Oracle app inventory which was the Oracle Warehouse Builder. Apparently the OWB installation puts that file there even though we set it in the OWB_HOME prior to the installation. Changing this file to point to the correct inventory was necessary. This is common issue if you have more than one Oracle application running on the same server under the same user. You need to update the oraInst.loc in /etc/ whenever you need to apply patches.

Second solution. This part had the Oracle Support scratching their heads. Still opatch lsinventory complained about ORACLE_HOME not being set – even though echo $ORACLE_HOME output was correct. They asked us to get UNIX admin to unmount the partition and remount it as readable. We tried that and a half other odd fixes without success. Finally Oracle support asked me to give them some trace dumps for them to analyze. The solution ended simply to make / readable by all!

How it was with ls -ld /
drwxr-x--x 35 root root 1024 Oct 1 00:52 /

After fix
drwxr-xr-x 35 root root 1024 Oct 1 01:52 /

Apparently the unix fellow who setup these servers for us thought it would be a security benefit if / was unreadable by all except root user. Unfortunately, Oracle cannot patch any of its applications with one-off patches when / unreadable by the oracle user.

How to Get Last Day of Last Month in Perl

Getting the last day of last month in perl can be a little tricky but I have just the solution for it.
First you find todays date using my favorite subroutine.

&get_date(0); #0 for number of days in past = today.

Second you take todays date and pass it to &get_date subroutine:

$current_day = $da; # $da is given a value by above subroutine.
&get_date($current_day); # Pass along todays date back to the get_date subroutine.

Now that you’ve passed the current day of month back to get_date, all the date values will be that of last day of last month.
Suppose today is September 7 2010

$yyyy = 2010
$mo = 08
$da = 31

Now lets put it all together.

#!/usr/bin/perl -w
use strict;
use Time::localtime; # &get_date needs this

# Date declarations
my $yyyy;
my $mo;
my $da;

my $current_days;    # Will be used to hold number of days in current month.
my $lastDayofLastMonth; # Variable for last day of the last month

# -----------------------------------------------------------------------
# Main
# -----------------------------------------------------------------------

&get_date_info;

print "Last day of last month was $lastDayofLastMonth \n";  # Will print out the last day of last month.

# -----------------------------------------------------------------------
# Figure last day of last month
# -----------------------------------------------------------------------
sub get_date_info {
    &get_date(0);
    $current_days = $da;
    &get_date($current_days);
    $lastDayofLastMonth = $yyyy . $mo . $da;

}

# -----------------------------------------------------------------------
# get date (my favorate subroutine!)
# -----------------------------------------------------------------------

sub get_date {
   my ($pastno) = @_; #Used to pass value for specific date in past.
   my $pastdate = localtime(time -(86400 * $pastno)); #pastno is # of days in past - set above
   $yyyy = $pastdate->year() + 1900;  # localtime->year() return years since 1900
   $mo = $pastdate->mon() + 1;        # localtime->mon() return 0-11,  Jan. - Dec.
   $da = $pastdate->mday();

   $mo = "0"."$mo"  if ($mo <= 9);
   $da = "0"."$da"  if ($da <= 9);

}
I don’t think this can be any simpler. If you have a much better way of doing this – PLEASE SHARE!

How to Setup Tomcat to Run on Port 80 Part 2

This is the second part to the “How to Setup Tomcat to Run on Port 80”.
Part 2
  1. Configure httpd.conf
  2. Add init.d Script
  3. Restart HTTPD

Configure The Domain’s httpd.conf

You will need to add some entries to the domain.conf file in /etc/httpd/conf.d/ or in httpd.conf file. I prefer to utilize the conf files in /etc/httpd/conf.d/ and use httpd.conf inclusion because it is so easy to mess up the httpd.conf file.

vi /etc/httpd/conf.d/yourdomain.conf

Add the following code before the </VirtualHost> entry.

<Location /WEB-INF/ > # (1) AllowOverride None deny from all </Location> <Location /META-INF/ > # (2) AllowOverride None deny from all </Location> JkAutoAlias /home/userX/tomcat/webapps # (3) JkMount /* tomcat1 # (4) JkMount /*/* tomcat1 # (5) JkUnMount /php/* tomcat1 # (6)
1. Security location entry for WEB-INF
2. Security for META-INF too
3. Kind of an auto invoker for webapps within webapps dir.
4. Invokes tomcat1 worker for root directory in url /
5. Invokes the worker for any directory deeper than /.  I have seen problems with recursive directories in this case and found this to solve the problem.
6. Perhaps you want apache to run a particular directory and not Tomcat. Just add UnMount to tell Apache to render instead.

Setup init.d Script

You’ll want an init.d script to start up your container when you boot up or reboot your server.
Create a file called tomcat.sh and place it in /etc/init.d/ and put in the following contents:

<div id="_mcePaste">#!/bin/sh
# Tomcat Startup Script

start() {

echo -n "Starting up tomcat1"
su username -c /home/username/tomcat/bin/start.sh

}
stop() {

echo -n "Stopping tomcat1"
su username -c /home/username/tomcat/bin/shutdown.sh

}

# See how we are called.
case "$1" in
        start)
                start
                ;;
        stop)
                stop
                ;;
        restart)
                stop
                start
                ;;
        *)
                echo $"Usage: tomcat {start|stop|restart}"
                exit
        esac
You will need to update the path to correctly point to your start and shutdown scripts. The su username -c is to insure that the system starts up your script as the unix user you have tomcat installed. Be sure to chmod the file to 755 so it can be executed when needed.

To complete this automation of the start up script, you will need to add a symlink in rc.3 in /etc/rc3.d/ as run level 3 is the default startup run level.
cd /etc/rc3.d/
ln -s ../init.d/tomcat.sh S99tomcatStartup
S is for startup and 99 is the process priority – you want tomcat to startup last.

Restart Apache

Finally best we restart httpd with service httpd restart to initialize in the updates we made to the domain.conf file.

Please leave comments. I am always interesting in learning better ways to do things.

How to Setup Tomcat to Run on Port 80 Part 1

This one is going to be a long post so I will break it up in two parts. My notes on this are brief but I will be more descriptive of this setup for the newbie benefit. Many people in the web host business have asked me how to setup tomcat or multiple instances of tomcat. I’ve given individuals an insight of what I learned years ago, but this is the first time I’ve posted for all to see. It’s actually very simple.

Steps for part 1

  1. Install Tomcat Connector
  2. worker.properties
  3. httpd_jk.conf
  4. Install Tomcat
  5. Configure server.xml
  6. Set JAVA_HOME
  7. start tomcat

Steps for part 2

  1. Configure domain’s httpd.conf file
  2. Add init script and symlink in rc.d
  3. restart httpd
You can start these steps in any order, but as I’ve always said, “Theres more than one way to skin a tomcat”.

Install Apache Tomcat Connector

Get latest release here: http://tomcat.apache.org/download-connectors.cgi The current version of this writing is tomcat-connectors-1.2.30-src.tar.gz and upload to your /root dir and untar.
tar -xvzf tomcat-connectors-1.2.30-src.tar.gz
cd into the tomcat-connectors-1.2.30-src/native and compile:
./configure --with-apxs=/usr/sbin/apxs  (or wherever your apxs is)
make
su -c 'make install';
You will then find mod_jk.so module in /usr/lib64/httpd/modules/
However, we’re not done with this yet. We still need to place a copy of workers.properties in /etc/httpd/conf/ and add mod_jk.so to apache by adding a file called httpd_jk.conf in /etc/httpd/conf.d where apache httpd.conf inclusion.
In the folder tomcat-connectors-1.2.30-src/conf/ you can find a copy of httpd_jk.conf and workers.properties.

workers.properties

cp workers.properties /etc/httpd/conf/
Vi into workers.properties and update the following:
Add a new worker after the manager worker entry as shown below for worker “tomcat1”:
worker.list=tomcat1 # (1)
worker.tomcat1.port=8007 # (2)
worker.tomcat1.host=localhost # (3)
worker.tomcat1.type=ajp13 # (4)
1. You specify the name of the new worker which in this case is tomcat1. This name can be any name, but must be unique among existing workers and with no spaces or special characters.
2. You specify the AJP port for the connector. 8007 is default. You can change this port here and in tomcat server.xml to utilize multiple instances of tomcat.
3. localhost since apache and tomcat are on the same server. Use IP or Hostname if apache and tomcat are on different servers which is common with clustering/load balancing.
4. ajp1.3 is currently the default worker type. ajp12 is deprecated and as of this writing ajp1.4 hasn’t been release.
Save the file.

httpd_jk.conf File

Now we need to copy the httpd_jk.conf to /etc/httpd/conf.d to enable mod_jk module in apache.
vi /etc/httpd/conf.d/httpd_jk.conf and validate the paths. Modify the path to workers with root path: /etc/httpd/conf/workers.properties and change logs/ to /var/log/httpd/. You may change them to whatever suits you best, but the above is my habit.
Now restart httpd and fix any errors that show up. This is end of installation of the apache connector.

Next step is to install Apache Tomcat Container

Download apache tomcat 6 from here: http://tomcat.apache.org/download-60.cgi
Untar it in the directory you wish to host it from. Should not be in /root or owned by root. I usually install it in /home/username/ and have it owned by that user.
tar -xvzf apache-tomcat-6.0.29.tar.gz
After you extract, create a symbolic link ‘tomcat’ to make things easier. Saves you from having to update other config if you ever upgrade tomcat.
ln -s apache-tomcat-6.0.29 tomcat

Configure server.xml

Update HTTP connector port if different from 8080. I usually defer to something else for security reasons.
Update the AJP 1.3 connector port if different from 8009 in workers.properties worker entry. This is where we match our entries in workers.properties so apache and tomcat know which worker to deal with when rendering pages.
Update defaultHost= from localhost to your domain (without www)
Update Host name to your domain name (without www).
Add alias to include www.domain.com
<Alias>www.domainX.com</Alias>

Add context path before closing host tag if you plan to use webapp other than ROOT:

<Context path=""
	docBase="/home/username/tomcat/webapps/webappName" crossContext="false" reloadable="true">
</Context>
Save file. Its good practice to make a backup of server.xml when you know this one is working.

Set JAVA_HOME

Did you install java? no? Best we do this now. Download the latest jdk 1.6 bin file wherever. I prefer the bin over the rpm installation because it allows me to setup multiple JDK versions. You can download and setup as user root in opt. Lets create folder in /opt/ called java and extract the file there.
First make jdk executable
chmod +x jdk-6u21-linux-i586.bin
Then expand. It will prompt you to agree to terms for it completes.
./jdk-6u21-linux-i586.bin
You will end up with a folder jdk-1.6_21
Now we need to set JAVA_HOME for your tomcat installation. There are 2 ways to do this, one is by setting export JAVA_HOME in your .bash_profile or you can set it up in catalina.sh file along with JAVA_OPTS. I prefer to use catalina.sh as sometimes I will have different jdk requirements for different containers on the same user. I find the latter method provides for better flexibility. Newer tomcat installations now have setclasspath.sh where JAVA_HOME can be defined. Also if you’re using the java service wrapper, then you will want to define JAVA_HOME in the wrapper.conf file in the conf directory of tomcat.
Add following after the comments in catalina.sh in the bin/ directory:
JAVA_HOME=/opt/java/jdk-1.6_21
export JAVA_HOME
At this point, you should be able to start your container.
cd tomcat/bin/
./catalina.sh run (if you want to watch the output - same as catalina.out log)
./startup.sh (or start it up and forget about it)
View tomcat/logs/catalina.out for any errors.
Now you can access your tomcat webpage or webapp using the HTTP port 8080 if its open.
Next post I will finish the rest of the httpd configuration so you can run your application on port 80.

How to get Todays Julian Date in Perl

There are 2 kinds of Julian dates. One that counts daily since time of Julius Caesar or the julian date based on current year. The latter seems to be more popular. Todays date is August 31st which is 2010243 or 10243 Julian date. This equals to 243 days since the first day of the year. There are some examples on the net but they get fancy with finding the leap year to taking actual julian date 2455440. The problem with many is they include the time of day as well and thus result in creating Perl code to clean it up. The method I use below comprises of just 3 subroutines, one to get todays date, another to calculate the days since the first of the year and finally puts together the last 2 digits of the year and the total days since the first.


#!/usr/bin/perl -w
use strict;
use Time::localtime; ### For get_date subroutine

## Declare variables for date handling
my $yyyy;  	
my $mo;  	
my $da;		


  &get_date(0); # Pass number of days in past to current or past date. 0 = today, 1 is yesterday etc.
   
   my $jyyy = $yyyy; #Since we need to make year unique for julian part.

   my $dayOfYear = &dayofyear($da,$mo,$jyyy); # Figure days of the year using subroutine.
   $jyyy -= 2000; # Removes the first 2 digits - likely a y2k gotcha if we used 1900 in 20th century.
   
   my $julianDate = $jyyy . $dayOfYear; # Put year and days together
   my $todaysDate = $yyyy . "/" . $mo . "/" . $da; # Put together todays date
   
   print "Todays date is $todaysDate \n";
   print "julianDate is $julianDate \n"; # Output 20100831 will = 10243


# -----------------------------------------------------------------------
# get number of days since first of the Year - Julian
# -----------------------------------------------------------------------

sub dayofyear {
    my ($day1,$month,$year)=@_;  # Passed todays date information
    my @cumul_d_in_m =
(0,31,59,90,120,151,181,212,243,273,304,334,365);
    my $doy=$cumul_d_in_m[--$month]+$day1;
    return $doy if $month < 2;
    return $doy unless $year % 4 == 0;
    return ++$doy unless $year % 100 == 0;
    return $doy unless $year % 400 == 0;
    return ++$doy;
}

# -----------------------------------------------------------------------
# get local time or past date.
# -----------------------------------------------------------------------

sub get_date {
   my $pastno = shift;
   my $pastdate = localtime(time -(86400 * $pastno));
   $yyyy = $pastdate->year() + 1900;  # localtime->year() return years since 1900
   $mo = $pastdate->mon() + 1;        # localtime->mon() return 0-11,  Jan. - Dec.
   $da = $pastdate->mday();

   $mo = "0"."$mo"  if ($mo <= 9);
   $da = "0"."$da"  if ($da <= 9);
   
}

My Best Practices with Perl

  • Add -w to shebang line to enable warnings. This helps point out flaws in your code during runtime.
  • use strict; and declare all variables with my. This forces you to keep your syntax clean and correct and avoid use of barewords.
  • Keep your code simple – KISS (keep is simple stupid). With Perl, there is more than 1 way to handle most objectives – try and keep it simple. I’ve experienced working with Perl scripts written by others and some people just have the knack of complicating the hell out of it. For example, one fellow wanted to find a file in a directory. Instead of just if (-e /path/filename), he or she read the entire directory into an ARRAY and then applied a grep to find the file. 10 lines of code to do what if (-e /path/filename.ext) can do!
  • Always comment your code and keep others in mind when doing so. Someone else may have to troubleshoot or take over your Perl script in the future. Also useful for when revisiting after 6 months. Sometimes I go back to my old Perl scripts and ask myself, “What the hell was I thinking!”.
  • Split up functions by subroutines. I never have one subroutine handle more than 1 function and they are usual designed to stand alone. Other subroutines will utilize other subroutines, but each remains 1 function. This practice allows me to keep my code organized and easier to update when needed. Its also great practice for code reuse.
  • Create modules of common functions. If you have several scripts utilizing a common function – create a module for them to share. I create modules for exception handling and logging, notifications, and many other things such as SecureTransfer or NDM functionality for file transfer in multiple scripts. This helps to reduce the time to update those features when needed. I have seen people duplicate common features in hundreds of scripts and find themselves having to modify each one when a change was needed. I’ve made this mistake myself and learned from it.
  • Learn and master regular expressions. This magic word, regex, is godsend! I use it all the time and its POWERFUL!
  • Practice code reuse. Build up a library of code that works. Saves a ton of time when scripting for new projects.

Migrating Oracle Warehouse builder and Data.

Oracle Warehouse Builder 10g currently does not support Data Guard that makes it easy to switch databases or migrate to another server and database. I ran into this similar limitation and had originally opted to redeploying all the mapping from the MDL export. This process what taking me between 4 to 6 hours because the Control Center GUI is extremely (not to mention painfully) slow. Finaly we worked with Oracle support to find a better solution to this problem. After some testing and several successful switched and migrations I have formulated the steps below.

Goal
Move SID and OWB installation and mappings to another server.

Pre-migration Steps

  1. Prepare 2 servers, app server and database server to have identical installation versions of database and OWB to the original systems.
  2. Have the DBA take a baseline of current setup using the owbcollect.sql that Oracle provides with OWB. This helps to compare after the installation of the new system is done. We do this step now because usually we end up putting it on standby mode before proceeding with the migration.
  3. Clone the SID and data and restore on new server. This process is beyond the scope here – usually the DBA takes care of this. I remember vaguely of the steps involved from my Oracle DBA training but I never had the opportunity to put it to practice.
  4. After the SID has been migrated and the database is running – start up OWB on the new server using the new SID information. OWB_HOME/owb/bin/unix/local_service_login.sh -startup /path/to/OWB  You can check to see if this is running by using ps -ef | grep OWB. You’ll receive a readout of the SID info at the end of the output.

Update the location credentials stored in the Control Center:

  1. Start the OWB Browser Listener. For Linux : OWB_home/owb/bin/unix/startOwbbInst.sh
  2. Start the Repository Browser. https://HOSTNAME:8999/owbb/RABLogin.uix?mode=design
  3. Connect to Warehouse Builder Browser as the Control Center Owner.
  4. Select “Locations Report” under Reports – Deployments
  5. Select the Location you would like to update
  6. Click on the “Unknown” link in the Validation column. (last column)
  7. In the “Connection Details” section of the Location Validation Report page, update the Host, Port and Service Name accordingly. Update to SID details to the new database server.
  8. Click on the “Update Details” button.
  9. Click on the “Test Location” button to check the location.
  10. Repeat steps above for each location that requires changes.
  11. Finally, Update the Control Center location “PlatformSchema” the same way as explained in steps above.

Update the Location Credentials stored in the Design Repository (DBA task)

  1. Using SQLPlus, connect as the warehouse builder repository owner.
  2. Execute the script UpdateLocation.sql (you can get script at Meta-link)
  3. SQL>@UpdateLocation.sql LOCATION_NAME HOST PORT SERVICE_NAME
  4. Do the above for each location that your OWB utilizes and provide the new server HOST, PORT and SERVICE_NAME(SID).

Note: LOCATION_NAME must be upper case or you will get a not found error.

Update the Control Centers Credentials stored in the Design Repository (DBA task)

  1. Using SQLPlus, connect as the warehouse builder repository owner.
  2. Execute the script UpdateControlCenter.sql attached to this document for each Control Center
  3. SQL>@UpdateControlCenter.sql CONTROL_CENTER_NAME HOST PORT SERVICE_NAME
  4. Do the above for each location that your OWB utilizes and provide the new server HOST, PORT and SERVICE_NAME(SID).
Note: LOCATION_NAME must be upper case or you will get a not found error.
Also: we were originally confused regarding the CONTROL_CENTER_NAME. This is almost always “DEFAULT_CONTROL_CENTER”.
Test the Migration
Log into the Design Center and then the control center to test a deploy or two to make sure all is working. Try one from staging and one from data warehouse. If successful deploy – then you should be good.
Finally run the baseline script on the new system using owbcollect.sql and compare with the original one you took prior to starting this process.Thats it – you’re done. You actually save about 5 hours with this process.

Troubleshooting
The only problem I’ve seen is when DBA was trying to update the locations credentials is that we we using the wrong location name. Or location that was being used in another environment but not with the system we were presently working with. Especially with confusion of lower environments having more locations for testing than production. Other than that I have not seen any other issues.

Calculating Time Lapse in Perl

I know there is a module that does this for you such as use Time::Elapse, but where I work, adding modules is a pain, and requires getting permission from upper management. I just find it easier to create the functionality myself.

My objective – to take 2 time stamps from a file and calculate the time lapse in minutes.


#!/usr/bin/perl -w
use strict;

my $start   = "02:01:16";
my $end     = "07:12:30";
my ($startHR, $startMIN) = split(/:/, $start); # split the time by colon to separate the hour and minutes.
my $startMinutes = ($startHR*60) + $startMIN; # Convert start time to minutes
my ($endHR, $endMIN) = split(/:/, $end); # start over with end time
my $endMinutes = ($endHR*60) + $endMIN;
my $range = $endMinutes - $startMinutes; # finally subtract the two.
print $range;

The result is 311 minutes.

My First Post

Technically this is not my first post. I have blogged in the past which you can find here: http://jklarsen.com/myblog.

I decided to start a new blog with entirely different focus on jklarsen.com. My goal with this blog is to share my experience with the technology field of IT. I am also deaf which provides some challenging obstacles at times and may be interesting how I get round those obstacles.

Currently I work for a large bank where I administer some of the Oracle applications as well as automate many of the nightly processing using bash, perl and autosys. I will dig back first to share some things I’ve learned, mistakes I’ve made and my thoughts on some of the technology that we use.