Prerequisites
Memory Requirements
As Red Hat Enterprise Linux 6 uses transparent Huge Pages and
configures automatically, allocate swap space to the amount of physical memory.
Disk Layout for the installation
|
File Types
|
Location
|
Mount Point
|
|
ORACLE_BASE
|
/u01/app/oracle
|
/u01
(not a shared filesystem)
|
|
ORACLE_HOME
|
/u01/app/oracle/product/11.2.0/dbhome_1
|
|
|
GRID_HOME
|
/u01/app/11.2.0/grid
|
|
|
OCR and Voting Disks
|
/ocrvote1/storage/ocr1
/ocrvote2/storage/ocr2
/ocrvote3/storage/ocr3
/ocrvote1/storage/vdsk1
/ocrvote2/storage/vdsk2
/ocrvote3/storage/vdsk3
|
/ocrvote1
/ocrvote2
/ocrvote3
|
|
Data Files, Controlfile 1
|
/u02/oradata/$ORACLE_SID
|
/u02/oradata/$ORACLE_SID
|
|
Redo Logs 1, Controlfile 2
|
/u03/oradata/$ORACLE_SID
|
/u03
|
|
Temp Files, Redo Logs 2, Control File 3
|
/u04/oradata/$ORACLE_SID
|
/u04
|
|
Fast Recovery Area
|
/u05/fra
|
/u05
|
All filesystems are shared across all the nodes in the cluster except
Oracle Binaries filesystem /u01. Put each database as its own storage volume.
Automatic
Setup
The Oracle RDBMS pre-install RPM sets kernel parameters, resource
limits for oracle account and installs the required RPMs. It also creates users
and groups using the next available IDs.
# yum update
# yum install oracle-rdbms-server-11gR2-preinstall
# usermod -g oinstall -G dba oracle
Check the RPM log file to review the system configuration changes.
/var/log/oracle-rdbms-server-11gR2-preinstall/results/orakernel.log
You can ignore the below manual settings if you installed the above
RPM.
Manual
Setup
Add the following lines to the "/etc/security/limits.conf"
file.
oracle soft nproc
2047
oracle hard nproc
16384
oracle soft nofile
4096
oracle hard nofile
65536
oracle soft stack
10240
Add or modify the OS Kernel Parameters in /etc/sysctl.conf
fs.suid_dumpable = 1
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 536870912
kernel.shmmni = 4096
# semaphores: semmsl, semmns, semopm, semmni
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586
To change the current kernel parameters.
# /sbin/sysctl -p
The following packages must be installed on RHEL 6.
binutils-2.20.51.0.2-5.11.el6
(x86_64)
compat-libcap1-1.10-1
(x86_64)
compat-libstdc++-33-3.2.3-69.el6
(x86_64)
compat-libstdc++-33-3.2.3-69.el6.i686
gcc-4.4.4-13.el6
(x86_64)
gcc-c++-4.4.4-13.el6
(x86_64)
glibc-2.12-1.7.el6
(i686)
glibc-2.12-1.7.el6
(x86_64)
glibc-devel-2.12-1.7.el6
(x86_64)
glibc-devel-2.12-1.7.el6.i686
ksh
libgcc-4.4.4-13.el6
(i686)
libgcc-4.4.4-13.el6
(x86_64)
libstdc++-4.4.4-13.el6
(x86_64)
libstdc++-4.4.4-13.el6.i686
libstdc++-devel-4.4.4-13.el6
(x86_64)
libstdc++-devel-4.4.4-13.el6.i686
libaio-0.3.107-10.el6
(x86_64)
libaio-0.3.107-10.el6.i686
libaio-devel-0.3.107-10.el6
(x86_64)
libaio-devel-0.3.107-10.el6.i686
make-3.81-19.el6
sysstat-9.0.4-11.el6
(x86_64)
Create Oracle User, Groups, and Directories.
# groupadd -g 501 oinstall
# groupadd -g 502 dba
# useradd -u 501 -g oinstall -G dba oracle
# mkdir -p /u01/app/11.2.0/grid
# mkdir -p /u01/app/oracle
# chown -R oracle:dba /u01
# chmod -R 775 /u01/
Set the password for oracle.
# passwd oracle
Deconfigure NTP so the Oracle Cluster Time Synchronization Service
(ctssd) can synchronize the times of the RAC nodes.
# service ntpd stop
Shutting down ntpd: [
OK ]
# chkconfig ntpd off
# mv /etc/ntp.conf /etc/ntp.conf.orig
# rm /var/run/ntpd.pid
Confirm the DNS is resolving SCAN VIPs. Do not configure SCAN VIP
addresses in the hosts file.
$ nslookup mycluster-scan
Server: dns.example.com
Address: 192.0.2.001
Name: mycluster-scan.example.com
Address: 192.0.2.201
Name: mycluster-scan.example.com
Address: 192.0.2.202
Name: mycluster-scan.example.com
Address: 192.0.2.203
For example, network interfaces for two node cluster. All addresses are fixed, not assigned by
DHCP. To enable VIP failover, the public, VIP, and SCAN addresses should be on
the same subnet.
|
Name
|
IP
|
Type
|
Resolved by
|
|
node1
|
192.0.2.101
|
public
|
DNS
|
|
node1-vip
|
192.0.2.104
|
virtual
|
DNS and hosts file
|
|
node1-priv
|
192.168.0.1
|
private
|
DNS or hosts file
|
|
node2
|
192.0.2.102
|
public
|
DNS
|
|
node2-vip
|
192.0.2.105
|
virtual
|
DNS and hosts file
|
|
node2-priv
|
192.168.0.2
|
private
|
DNS or hosts file
|
|
mycluster-scan
|
192.0.2.201
|
virtual
|
DNS
|
|
mycluster-scan
|
192.0.2.202
|
virtual
|
DNS
|
|
mycluster-scan
|
192.0.2.203
|
virtual
|
DNS
|
# cat /etc/hosts
# Public IPs
192.0.2.101 orarac1.localdomain orarac1
192.0.2.102 orarac2.localdomain orarac2
# Private IPs
192.168.0.1 orarac1-priv.localdomain orarac1-priv
192.168.0.2 orarac2-priv.localdomain
orarac2-priv
# Virtual IPs
192.0.2.104 orarac1-vip.localdomain orarac1-vip
192.0.2.105 orarac2-vip.localdomain orarac2-vip
# SCAN Virtual IPs must be configured in DNS
192.0.2.201 orarac-scan.localdomain orarac-scan
192.0.2.202 orarac-scan.localdomain orarac-scan
192.0.2.203 orarac-scan.localdomain orarac-scan
Download
the software from My Oracle Support
Installation Types
and Associated Zip files.
|
Installation Type
|
Zip File
|
|
Oracle Database
includes RAC
|
p13390677_112040_Linux-x86-64_1of7.zip
p13390677_112040_Linux-x86-64_2of7.zip
|
|
Oracle Grid
Infrastructure includes Clusterware, ASM, and Restart
|
p13390677_112040_Linux-x86-64_3of7.zip
|
|
Oracle Database
Client
|
p13390677_112040_Linux-x86-64_4of7.zip
|
|
Oracle Gateways
|
p13390677_112040_Linux-x86-64_5of7.zip
|
|
Oracle Examples
|
p13390677_112040_Linux-x86-64_6of7.zip
|
|
Deinstall
|
p13390677_112040_Linux-x86-64_7of7.zip
|
Log on to My
Oracle Support at https://support.oracle.com
Click Patches and
Updates.
In the patch search
section, click the Search tab.
Enter Patch Number
and select Platform.
Click Search
In the Patch
Search Results page, Click Download.
From the file
download window, click each file name to download the selected software.
$ cd /u01/app/oracle
$ mkdir software
$ cd software
$ unzip p13390677_112040_Linux-x86-64_1of7.zip
$ unzip p13390677_112040_Linux-x86-64_2of7.zip
$ unzip p13390677_112040_Linux-x86-64_3of7.zip
Without cvuqdisk, the Cluster Verification Utility (CVU) cannot find
shared disks, and you receive a "Package cvuqdisk not installed"
error when you run the Cluster Verification Utility.
# cd grid/rpm
# rpm -ivh cvuqdisk-1.0.7-1.rpm
Preparing...
########################################### [100%]
Using default group oinstall to install package
1:cvuqdisk
########################################### [100%]
Environment
Set the environment in /home/oracle/.bash_history
# Oracle Settings
ORACLE_BASE=/u01/app/oracle
export ORACLE_BASE
ORACLE_HOME=$ORACLE_BASE/product/11.2.0/dbhome_1
GRID_HOME=/u01/app/11.2.0/grid
ORACLE_SID=RAC1
export ORACLE_HOME GRID_HOME ORACLE_SID
PATH=$ORACLE_HOME/bin:/usr/sbin:$PATH:.
export PATH
LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export LD_LIBRARY_PATH
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
export CLASSPATH
if [ $USER = "oracle" ]; then
if [ $SHELL =
"/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi
Login as oracle and start Oracle Universal Installer.
$ cd grid
$ ./runInstaller
Starting Oracle Universal Installer...
Checking Temp space: must be greater than 120 MB. Actual 7204 MB Passed
Checking swap space: must be greater than 150 MB. Actual 4063 MB Passed
Checking monitor: must be configured to display at least 256
colors. Actual 16777216 Passed
Preparing to launch Oracle Universal Installer from
/tmp/OraInstall2014-01-03_07-30-39PM. Please wait ...
On Download Software Updates screen, select Skip software updates.
On Select Installation Option screen, select Install and Configure Grid
Infrastructure for a Cluster.
On Select Installation Type screen, select Advanced Installation.
The Selected Language is English. Click Next to continue.
On Grid Plug and Play Information page, enter Cluster Name, SCAN Name,
and SCAN Port. Uncheck Configure GNS as the SCAN VIPs are configured in DNS.
Click Next to validate SCAN information.
On Cluster Node Information screen, click on Add button.
Enter the details for second node.
Click on SSH Connectivity… to establish Paswordless SSH connectivity
between the selected nodes. Enter the password for oracle user and setup the
SSH connection.
Click Next to validate node readiness.
On Specify Network Interface Usage screen, check Public and Private
networks are specified correctly. Click Next to validate public and private
addresses across cluster nodes.
On Storage Option Information screen, select Shared File System.
On OCR Storage Options screen, It fetches shared partitions for OCR
File locations. Select Normal Redundancy and enter OCR File Locations
correctly. Click Next to validate OCR locations.
On Voting Disk Storage Option screen, it fetches shared volumes for
Voting Disk File Locations. Select Normal Redundancy and enter Voting Disk File
Locations correctly. Click next to validate voting disk locations.
On Failure Isolation Support screen, accept default Failure Isolation
Support i.e. Do not use Intelligent Platform Management Interface (IPMI).
On Privileged Operation System Groups screen, select the Privileged
Operating System Groups. Click Next.
As dba group is selected for all, this group will not allow job role
separation. A warning will appear. Click Yes.
On Specify Installation Location screen, the Oracle Base path appears
by default. Accept the default values or enter the Grid Infrastructure home
directory path in which you want to install Grid Infrastructure. The directory
path should not contain spaces. Click Next to check specified location on
remote host.
Inventory Directory appears by
default for the first installation on this host. Accept default values or
specify the full path of Oracle Inventory Directory and OS group.
This screen checks that the system meets the minimum requirements for
the installation and configuration.
Correct any errors that Oracle Universal Installer may have found.
If there are no issues, you will directly move to the Summary screen.
On Summary screen displays Global Settings, Grid
Infrastructure Settings, and Strorage Information. Click Install.
The Install Product page provides status so that you can monitor
installation progress as Installer continues.
As root, execute the configuration scripts on each node.
On node 1:
# /u01/app/11.2.0/grid/root.sh
Performing root user operation for Oracle 11g
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/11.2.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to
/usr/local/bin ...
Copying coraenv to /usr/local/bin
...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
yUsing configuration parameter file:
/u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
Installing Trace File Analyzer
OLR initialization - successful
root wallet
root wallet cert
root cert export
peer wallet
profile reader wallet
pa wallet
peer wallet keys
pa wallet keys
peer cert request
pa cert request
peer cert
pa cert
peer root cert TP
profile reader root cert TP
pa root cert TP
peer pa cert TP
pa peer cert TP
profile reader pa cert TP
profile reader peer cert TP
peer user cert
pa user cert
Adding Clusterware entries to upstart
CRS-2672: Attempting to start 'ora.mdnsd' on 'orarac1'
CRS-2676: Start of 'ora.mdnsd' on 'orarac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'orarac1'
CRS-2676: Start of 'ora.gpnpd' on 'orarac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'orarac1'
CRS-2672: Attempting to start 'ora.gipcd' on 'orarac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'orarac1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'orarac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'orarac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'orarac1'
CRS-2676: Start of 'ora.diskmon' on 'orarac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'orarac1' succeeded
clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Now formatting voting disk: /ocrvote1/storage/vdsk1.
Now formatting voting disk: /ocrvote2/storage/vdsk2.
Now formatting voting disk: /ocrvote3/storage/vdsk3.
CRS-4603: Successful addition of voting disk /ocrvote1/storage/vdsk1.
CRS-4603: Successful addition of voting disk /ocrvote2/storage/vdsk2.
CRS-4603: Successful addition of voting disk /ocrvote3/storage/vdsk3.
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 5ab6de93bbe34fa0bfdef769ebf332bf
(/ocrvote1/storage/vdsk1) []
2. ONLINE 93cc46a0713a4febbff737f4d375f53b
(/ocrvote2/storage/vdsk2) []
3. ONLINE 58a96a6c3ae44fa3bfb4fc72c47a5820
(/ocrvote3/storage/vdsk3) []
Located 3 voting disk(s).
Preparing packages for installation...
cvuqdisk-1.0.9-1
Configure Oracle Grid
Infrastructure for a Cluster ... succeeded
On node 2:
# /u01/app/11.2.0/grid/root.sh
Performing root user operation for Oracle 11g
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/11.2.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to
overwrite.
The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
Copying oraenv to
/usr/local/bin ...
The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
Copying coraenv to
/usr/local/bin ...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
y
Using configuration parameter file:
/u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
Installing Trace File Analyzer
OLR initialization - successful
Adding Clusterware entries to upstart
CRS-4402: The CSS daemon was started in exclusive mode but found an
active CSS daemon on node orarac1, number 1, and is terminating
An active cluster was found during exclusive startup, restarting to
join the cluster
Preparing packages for installation...
cvuqdisk-1.0.9-1
Configure Oracle Grid
Infrastructure for a Cluster ... succeeded
Once the scripts have completed, click OK on Execute Configuration
Scripts screen.
It is expected that the verification phase will fail with an error
related to SCAN. This install has not used DNS for SCAN VIPs. Click Yes to
continue.
NFO: TCP connectivity to SCAN Listeners exists on all cluster nodes
INFO: Checking name resolution setup for
"orarac-scan.localdomain"...
INFO: Checking integrity of name service switch configuration file
"/etc/nsswitch.conf" ...
INFO: All nodes have same "hosts" entry defined in file
"/etc/nsswitch.conf"
INFO: Check for integrity of name service switch configuration file
"/etc/nsswitch.conf" passed
INFO: ERROR:
INFO: PRVG-1101 : SCAN name "orarac-scan.localdomain" failed
to resolve
INFO: ERROR:
INFO: PRVF-4657 : Name resolution setup check for
"orarac-scan.localdomain" (IP address: 192.168.1.113) failed
INFO: ERROR:
INFO: PRVF-4657 : Name resolution setup check for
"orarac-scan.localdomain" (IP address: 192.168.1.114) failed
INFO: ERROR:
INFO: PRVF-4657 : Name resolution setup check for
"orarac-scan.localdomain" (IP address: 192.168.1.115) failed
INFO: ERROR:
INFO: PRVF-4664 : Found inconsistent name resolution entries for SCAN
name "orarac-scan.localdomain"
INFO: Verification of SCAN VIP and Listener setup failed
Click Close to exit the installer.
Check state of the resources on the cluster.
$ crsctl status resource -t
--------------------------------------------------------------------------------
NAME TARGET STATE
SERVER
STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
ONLINE ONLINE
orarac1
ONLINE ONLINE
orarac2
ora.asm
OFFLINE
OFFLINE orarac1 Instance Shutdown
OFFLINE
OFFLINE orarac2
ora.gsd
OFFLINE
OFFLINE orarac1
OFFLINE
OFFLINE orarac2
ora.net1.network
ONLINE ONLINE
orarac1
ONLINE ONLINE
orarac2
ora.ons
ONLINE ONLINE
orarac1
ONLINE ONLINE
orarac2
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE
ONLINE orarac2
ora.LISTENER_SCAN2.lsnr
1 ONLINE
ONLINE orarac1
ora.LISTENER_SCAN3.lsnr
1 ONLINE
ONLINE orarac1
ora.cvu
1 ONLINE
ONLINE orarac1
ora.oc4j
1 ONLINE
ONLINE orarac1
ora.orarac1.vip
1 ONLINE
ONLINE orarac1
ora.orarac2.vip
1 ONLINE
ONLINE orarac2
ora.scan1.vip
1 ONLINE
ONLINE orarac2
ora.scan2.vip
1
ONLINE ONLINE orarac1
ora.scan3.vip
1 ONLINE
ONLINE orarac1
Check cluster services.
$ crsctl check cluster -all
**************************************************************
orarac1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
orarac2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
Check remote listener SCAN1 status.
$ lsnrctl status LISTENER_SCAN1
LSNRCTL for Linux: Version 11.2.0.4.0 - Production on 07-JAN-2014
09:09:37
Copyright (c) 1991, 2013, Oracle.
All rights reserved.
Connecting to
(DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN1)))
STATUS of the LISTENER
------------------------
Alias LISTENER_SCAN1
Version TNSLSNR
for Linux: Version 11.2.0.4.0 - Production
Start Date
07-JAN-2014 08:43:19
Uptime 0 days
0 hr. 26 min. 19 sec
Trace Level off
Security ON:
Local OS Authentication
SNMP OFF
Listener Parameter File
/u01/app/11.2.0/grid/network/admin/listener.ora
Listener Log File
/u01/app/11.2.0/grid/log/diag/tnslsnr/orarac2/listener_scan1/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_SCAN1)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.0.2.201)(PORT=1521)))
The listener supports no services
The command completed successfully
Check local listener status on node 1.
$ lsnrctl status LISTENER
LSNRCTL for Linux: Version 11.2.0.4.0 - Production on 07-JAN-2014
08:56:39
Copyright (c) 1991, 2013, Oracle.
All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))
STATUS of the LISTENER
------------------------
Alias
LISTENER
Version TNSLSNR
for Linux: Version 11.2.0.4.0 - Production
Start Date
07-JAN-2014 08:40:49
Uptime 0 days
0 hr. 15 min. 50 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File
/u01/app/11.2.0/grid/network/admin/listener.ora
Listener Log File
/u01/app/oracle/diag/tnslsnr/orarac1/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.0.2.101)(PORT=1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.0.2.104)(PORT=1521)))
The listener supports no services
The command completed successfully
Check local listener status on node 2.
$ lsnrctl status LISTENER
LSNRCTL for Linux: Version 11.2.0.4.0 - Production on 07-JAN-2014
09:01:06
Copyright (c) 1991, 2013, Oracle.
All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))
STATUS of the LISTENER
------------------------
Alias
LISTENER
Version TNSLSNR
for Linux: Version 11.2.0.4.0 - Production
Start Date
07-JAN-2014 08:43:19
Uptime 0 days 0 hr. 17 min. 48 sec
Trace Level off
Security ON:
Local OS Authentication
SNMP OFF
Listener Parameter File
/u01/app/11.2.0/grid/network/admin/listener.ora
Listener Log File /u01/app/oracle/diag/tnslsnr/orarac2/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.0.2.102)(PORT=1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.0.2.105)(PORT=1521)))
The listener supports no services
The command completed successfully
No comments:
Post a Comment