一个管理员要管理多台机器，并且服务器的密码要定期修改，尤其在有很多台服务器的情况下，这是一个艰巨的任务，但理用公私钥，就可以很轻松的维护服务器的密码问题，我们完全可以让Linux服务器的登陆认证基于公私钥而不是密码，如下以SecureCRT为工具所示：
1.使用SecureCRT创建私钥和公钥.
SecureCRT: Quick Connect -> Authentiation -> Public Key -> Properties -> Create Identity File -> DSA/RSA -> Set Passphrase -> Done这个时候在指定目录会生成两个文件，例如，私钥my_rsa和公钥my_rsa.pub
2.将公钥 my_rsa.pub 传到linux服务器，将SSH2兼容格式的公钥转换成为Openssh兼容格式
# ssh-keygen -i -f Identity.pub >> /root/.ssh/authorized_keys2
# chmod 600 /root/.ssh/authorized_keys2
3.在SecureCRT里面设置登录模式为PublicKey，并选择刚刚创建的my_rsa文件作为私钥
4.由于已经设置了密钥登录，原来的密码登录就完全可以去掉
# vi /etc/ssh/sshd_config
Protocol 2 /仅允许使用SSH2
PubkeyAuthentication yes /*启用PublicKey认证
AuthorizedKeysFile .ssh/authorized_keys2 /*PublicKey文件路径
PasswordAuthentication no /*禁止密码验证登录
以上步骤是使用SecureCRT生成的密钥对来进行登录验证的，其实也可以在服务器上使用ssh-keygen命令生成的密钥，同样在生成密钥对之后，将格式转换成SecureCRT 使用的SSH2格式
星期日, 四月 29, 2007
redhat linux下安装Veritas netbackup软件实录
1．安装master server
（1）运行安装程序
# /cdrom1/install
VERITAS Installation Script
Copyright 1993 - 2004 VERITAS Software Corporation, All Rights Reserved.
Installation Options
1 NetBackup
2 NetBackup Client Software
q To quit from this script
Choose an option [default: q]: 1
The NetBackup and Media Manager software is built for use on LINUX hardware.
Do you want to install NetBackup and Media Manager files? [y,n] (y) y
NetBackup and Media Manager are normally installed in /usr/openv.
Is it OK to install in /usr/openv? [y,n] (y) y
The Linux clients will be loaded.
Do you want to load any other NetBackup clients onto the server? [y,n] (y) n
开始安装
Reading client binaries for /usr/openv/netbackup/client/Linux
Linux/
......
+ /bin/cp ./fsanalyze /usr/openv/netbackup/bin/fsanalyze
A NetBackup Server or Enterprise Server license key is needed
for installation to continue.
Enter license key: (输入master licence)
:
NetBackup Enterprise Server Base product with the following features enabled:
Open File Backup
Remote Client Support
Robotic Library Sharing Support
Remote Media Server Support
Microsoft RSM Robotic Libraries
ADIC DAS/SDLC Robotic Libraries
IBM ATL Robotic Libraries
Fujitsu LMF Robotic Libraries
StorageTek ACS Robotic Libraries
Inline Tape Copy
has been registered.
All additional keys should be added at this time.
Do you want to add additional license keys now? [y,n] (y) n
是否增加其他licence，可以选y继续输入或选n等安装完后在管理界面上再添加
If this machine will be using a different network interface than the
default (gzdb1), the name of the preferred interface should be used
as the configured server name. If this machine will be part of a
cluster, the virtual name should be used as the configured server name.
The domainname of your server appears to be "(none)".
You may choose to use this domainname in your configured NetBackup server
name, or simply use "gzdb1" as the configured NetBackup server name.
Would you like to use "gzdb1.(none)" as the configured
name of the NetBackup server? [y,n] (y) n （选n）
Enter the name of the NetBackup server: gzdb1 （手工填写本机名，可不用域名）
Is gzdb1 the master server? [y,n] (y) （问本机是否是master server,选y）
Do you have any media servers? [y,n] (n) （问有无其他media server，暂无选n）
......
安装过程会在/etc/services增加服务
......
To change these entries modify the file /tmp/services.ov_edited.27459
and enter when ready to continue: （按回车继续）
......
Sending SIGHUP to xinetd process.
Reloading configuration: [ OK ] （系统的xinetd服务必须已启动）
......
安装过程会在/etc/rc.d增加系统启动和关闭服务脚本
......
Enter which host will store the global device database.
(default: gzdb1): （问全局库的主机，如默认显示是master server就按回车，否及手工输入）
Starting the NetBackup database manager process (bpdbm).
Do you want to create policy and schedule examples that you can view or use
when you are configuring your own policies and schedules? [y,n] (y) （问是否创建备份策略调度列表及事例，选y）
......
安装过程在创建policy and schedule
......
Do you want to start the Media Manager device daemon processes? [y,n] (y) （问是否启动Media Manager服务，选y）
Starting the Media Manager device daemon processes.
Do you want to start the NetBackup bprd process so
backups and restores can be initiated? [y,n] (y) （问备份恢复服务是否启动，选y）
Starting the NetBackup request daemon process (bprd).
Done executing NB.inst
VERITAS Installation Script
Copyright 1993 - 2004 VERITAS Software Corporation, All Rights Reserved.
Installation Options
1 NetBackup
2 NetBackup Client Software
q To quit from this script
Choose an option [default: q]: （安装完毕，选q退出）
（2） 编辑.bash_profile
在root目录下的.bash_profile里手工加入Veritas的命令路径，便于日常命令使用。
加入：
PATH=$PATH:$HOME/bin:/usr/openv/netbackup/bin:/usr/openv/netbackup/bin/admincmd:/usr/openv/netbackup/bin/goodies:/usr/openv
/volmgr/bin
保存退出
执行使生效：#. ./.bash_profile
执行bpps -a查看进程，应有以下进程
# bpps -a
NB Processes
------------
root 27633 1 0 11:23 pts/0 00:00:00 /usr/openv/netbackup/bin/bpdbm
root 27906 1 0 11:23 pts/0 00:00:00 /usr/openv/netbackup/bin/bprd
root 27635 27633 0 11:23 pts/0 00:00:00 /usr/openv/netbackup/bin/bpjobd
MM Processes
------------
root 27613 1 0 11:23 pts/0 00:00:00 /usr/openv/volmgr/bin/vmd
（3）编辑/usr/openv/netbackup/bp.conf文件
根据主机脚色加入以下信息
SERVER = gzdb1
SERVER = gzwebdb
SERVER = gzwebdbbak
CLIENT_NAME = gzdb1
MEDIA_SERVER = gzwebdb
MEDIA_SERVER = gzwebdbbak
......
注：SERVER--〉包括master server 和media server；
CLIENT_NAME--〉本机及要通过网络备到本机的主机；
MEDIA_SERVER--〉包括master server 和media server。
（4）安装补丁
安装以下所补丁
NB_51_4_M_280420.linux.tar
NB_CLT_51_4_M_280431.tar
NB_ORA_51_4_M_280463.linux2.4.tar
1）将文件放在一临时目录下，全部用tar接压；
2）用netbackup stop停止netbackup服务；
3）运行安装脚本；
#./Vrts_pack.install
There are 3 packs available in /data/veritas_inst:
(* denotes installed pack)
NB_51_4_M
NB_CLT_51_4_M
NB_ORA_51_4_M
Enter pack name (or q) [q]: NB_51_4_M （在列表里选择第一个补丁）
Pack NB_51_4_M requires the following pack levels(s) :
NB_CLT_51_4_M.
Pack NB_51_4_M will not be installed at this time.
Installing required pack, NB_CLT_51_4_M, now.
Pack NB_CLT_51_4_M co-requires the following pack level(s) :
NB_51_4_M.
Install pack NB_CLT_51_4_M Wed Feb 15 14:56:14 CST 2006 Rev. 1.19.4.23
Performing install of Linux/RedHat2.4 client binaries locally.
Running preinstall script.
......
安装程序会将“NB_CLT_51_4_M”一并安装
......
------------------------------------------------
Installation of pack NB_51_4_M completed Wed Feb 15 14:56:14 CST 2006 Rev. 1.19.4.23.
------------------------------------------------
There are 3 packs available in /data/veritas_inst:
(* denotes installed pack)
NB_51_4_M *
NB_CLT_51_4_M *
NB_ORA_51_4_M
Enter pack name (or q) [q]: （打*号表示已安装的补丁，master server不需安装NB_ORA_51_4_M，选q退出）
4）执行netbackup start重新启动服务。
2．安装media server
（1）运行安装程序
# /cdrom1/install
VERITAS Installation Script
Copyright 1993 - 2004 VERITAS Software Corporation, All Rights Reserved.
Installation Options
1 NetBackup
2 NetBackup Client Software
q To quit from this script
Choose an option [default: q]: 1
The NetBackup and Media Manager software is built for use on LINUX hardware.
Do you want to install NetBackup and Media Manager files? [y,n] (y) y
NetBackup and Media Manager are normally installed in /usr/openv.
Is it OK to install in /usr/openv? [y,n] (y) y
The Linux clients will be loaded.
Do you want to load any other NetBackup clients onto the server? [y,n] (y) n
开始安装
Reading client binaries for /usr/openv/netbackup/client/Linux
Linux/
......
+ /bin/cp ./fsanalyze /usr/openv/netbackup/bin/fsanalyze
A NetBackup Server or Enterprise Server license key is needed
for installation to continue.
Enter license key: (输入master licence)
:
NetBackup Enterprise Server Base product with the following features enabled:
Open File Backup
Remote Client Support
Robotic Library Sharing Support
Remote Media Server Support
Microsoft RSM Robotic Libraries
ADIC DAS/SDLC Robotic Libraries
IBM ATL Robotic Libraries
Fujitsu LMF Robotic Libraries
StorageTek ACS Robotic Libraries
Inline Tape Copy
has been registered.
All additional keys should be added at this time.
Do you want to add additional license keys now? [y,n] (y) n
是否增加其他licence，可以选y继续输入或选n等安装完后在管理界面上再添加
If this machine will be using a different network interface than the
default (gzwebdb), the name of the preferred interface should be used
as the configured server name. If this machine will be part of a
cluster, the virtual name should be used as the configured server name.
The domainname of your server appears to be "(none)".
You may choose to use this domainname in your configured NetBackup server
name, or simply use "gzwebdb" as the configured NetBackup server name.
Would you like to use "gzwebdb.(none)" as the configured
name of the NetBackup server? [y,n] (y) n （选n）
Enter the name of the NetBackup server: gzwebdb （手工填写本机名，可不用域名）
Is gzdb1 the master server? [y,n] (n) （问本机是否是master server,选n）
What is the fully qualified name of the master server? gzdb1 （在此输入master server的主机名，如“gzdb1”）
......
安装过程会在/etc/services增加服务
......
To change these entries modify the file /tmp/services.ov_edited.27459
and enter when ready to continue: （按回车继续）
......
Sending SIGHUP to xinetd process.
Reloading configuration: [ OK ] （系统的xinetd服务必须已启动）
......
安装过程会在/etc/rc.d增加系统启动和关闭服务脚本
......
Enter global device database host (default: gzdb1): （问全局库的主机，如默认显示是master server就按回车，否及手工输入）
Do you want to start the Media Manager device daemon processes? [y,n] (y) （问是否启动Media Manager服务，选y）
Starting the Media Manager device daemon processes.
Done executing NB.inst
VERITAS Installation Script
Copyright 1993 - 2004 VERITAS Software Corporation, All Rights Reserved.
Installation Options
1 NetBackup
2 NetBackup Client Software
q To quit from this script
Choose an option [default: q]: （安装完毕，选q退出）
（2）编辑.bash_profile
在root目录下的.bash_profile里手工加入Veritas的命令路径，便于日常命令使用。
加入：
PATH=$PATH:$HOME/bin:/usr/openv/netbackup/bin:/usr/openv/netbackup/bin/admincmd:/usr/openv/netbackup/bin/goodies:/usr/openv
/volmgr/bin
保存退出
执行使生效：#. ./.bash_profile
执行bpps -a查看进程，应有以下进程
# bpps -a
NB Processes
------------
MM Processes
------------
root 27613 1 0 11:23 pts/0 00:00:00 /usr/openv/volmgr/bin/vmd
（3）编辑/usr/openv/netbackup/bp.conf文件
根据主机脚色加入以下信息
SERVER = gzdb1
SERVER = gzwebdb
SERVER = gzwebdbbak
CLIENT_NAME = gzdb1
MEDIA_SERVER = gzwebdb
MEDIA_SERVER = gzwebdbbak
......
注：SERVER--〉包括master server 和media server；
CLIENT_NAME--〉本机及要通过网络备到本机的主机；
MEDIA_SERVER--〉包括master server 和media server。
（4）安装oracle agent
因为media server作为client端，需要在线备份oracle数据库，所以要安装database agent用以数据库的备份。
1） 进入加载database agent的光盘或IOS目录
cd /cdrom2
2)运新安装程序
# ./install
VERITAS Installation Script
Copyright 1993 - 2004 VERITAS Software Corporation, All Rights Reserved.
Installation Options
1 NetBackup Add-On Product Software
2 NetBackup Database Agent Software
q To quit from this script
Choose an option [default: q]: 2
**********
There are two ways to install database agent software.
1. Remote Installation: Loads the software on a master server with
the intent of pushing database software out to affected clients.
2. Local Installation: Loads and installs the software only to this
local machine.
**********
Do you want to do a local installation? [y,n] (n) 2
You have entered an invalid option.
Do you want to do a local installation? [y,n] (n) y
**********
NetBackup Database Agent Installation
Choose the Database Agents you wish to install
one at a time or select Install All Database Agents.
1) NetBackup for DB2
2) NetBackup for Informix
3) NetBackup for Lotus Notes
4) NetBackup for Oracle
5) NetBackup for SAP
6) NetBackup for Sybase
7) Install All Database Agents
q) Done Selecting Agents
x) Exit from this Script
Select the Oracle option if upgrading any of the
following: Oracle, Oracle on UNIX Advanced BLI Agent,
Oracle ServerFree Agent or Database Archiver.
The packages have been combined.
Choose an option: 4 （选4，安装oracle类型）
Choose an option: q （如没其他类型的数据库，默认q退出选择状态）
You have chosen to install these Database Agents:
NetBackup for Oracle
Is this list correct? [y,n] (y) （显示当前选择的database 类型，选y开始安装）
**********
Of the agents selected, the following are supported
on this platform and will be installed:
Oracle
Loading the Database Agent packages into the
/usr/openv/netbackup/dbext directory and installing.
**********
Installing NetBackup for Oracle
……安装程序在安装文件
NetBackup for Oracle installation completed.
**********
Execution of ./install_dbext is complete.
Execution of ./install is complete.
File /tmp/install_options_trace.27635 contains a trace of this install.
VERITAS Installation Script
Copyright 1993 - 2004 VERITAS Software Corporation, All Rights Reserved.
Installation Options
1 NetBackup Add-On Product Software
2 NetBackup Database Agent Software
q To quit from this script
Choose an option [default: q]:
退出完成安装
（5）安装补丁
安装以下所补丁
NB_51_4_M_280420.linux.tar
NB_CLT_51_4_M_280431.tar
NB_ORA_51_4_M_280463.linux2.4.tar
1）将文件放在一临时目录下，全部用tar接压；
2）用netbackup stop停止netbackup服务；
3）运行安装脚本；
#./Vrts_pack.install
There are 3 packs available in /data/veritas_inst:
(* denotes installed pack)
NB_51_4_M
NB_CLT_51_4_M
NB_ORA_51_4_M
Enter pack name (or q) [q]: NB_51_4_M （在列表里选择第一个补丁）
Pack NB_51_4_M requires the following pack levels(s) :
NB_CLT_51_4_M.
Pack NB_51_4_M will not be installed at this time.
Installing required pack, NB_CLT_51_4_M, now.
Pack NB_CLT_51_4_M co-requires the following pack level(s) :
NB_51_4_M.
Install pack NB_CLT_51_4_M Wed Feb 15 14:56:14 CST 2006 Rev. 1.19.4.23
Performing install of Linux/RedHat2.4 client binaries locally.
Running preinstall script.
......
安装程序会将“NB_CLT_51_4_M”一并安装
......
------------------------------------------------
Installation of pack NB_51_4_M completed Wed Feb 15 14:56:14 CST 2006 Rev. 1.19.4.23.
------------------------------------------------
There are 3 packs available in /data/veritas_inst:
(* denotes installed pack)
NB_51_4_M *
NB_CLT_51_4_M *
NB_ORA_51_4_M
Enter pack name (or q) [q]: NB_ORA_51_4_M （在列表里选择ORA补丁）
Install pack NB_ORA_51_4_M Wed Feb 15 14:56:14 CST 2006 Rev. 1.19.4.23
Please pick one of the following methods for installing this database pack.
1. Remote Installation: Loads the software on a master server with
the intent of pushing the pack out to affected clients.
2. Local Installation: Loads and installs the pack only to this
local machine.
Enter option (or q) [q]: 2
Saving pre-existing binaries.
Running preinstall script.
......
安装程序会将“NB_ORA_51_4_M”补丁
......
------------------------------------------------
Installation of pack NB_ORA_51_4_M completed Wed Feb 15 15:16:14 CST 2006 Rev. 1.19.4.23.
------------------------------------------------
There are 3 packs available in /data/veritas_inst:
(* denotes installed pack)
NB_51_4_M *
NB_CLT_51_4_M *
NB_ORA_51_4_M *
Enter pack name (or q) [q]: （打*号表示已安装的补丁，选q退出）
4）执行netbackup start重新启动服务。
5）用Veritas链接数据库
Veritas必须和数据库进行链接后才可以正常使用，使用veritas自带脚本（位置/usr/openv/netbackup/bin/oracle_link）进行数据库链接
，直接运行即可：
/usr/openv/netbackup/bin/oracle_link
三、配置Veritas
1．图形管理界面调用
1）用图形工具，如VNC打开master server图形界面，新建一个终端窗口。
2）运行/usr/openv/netbackup/bin/jnbSA命令出现“Netbackup Admin Console”界面。
2．配置Storage Units
在“Storage Units”处配置master server和media server的备份设备。
master server:
例（主机名为gzdb1，备份目录为/backup）
按右键选new
"Storage unit name"输入gzdb1-disk
"NerBackup media server"输入gzdb1
"Storage unit tupe"选disk
"Absolute pathname to directory"输入/backup
按"OK"退出
media server:
例（主机名为gzwebdb，备份目录为/backup）
按右键选new
"Storage unit name"输入gzwebdb-disk
"NerBackup media server"输入gzwebdb
"Storage unit tupe"选disk
"Absolute pathname to directory"输入/backup
按"OK"退出
......
如此将所有media server的主机都新建起来。
3．配置Veritas的catalog信息库备份
1）点左窗口的最上方的“gzdb1(Master Server)”->再选右边的“Configure the Catalog Backup”配置向导；
2）按提示操作选择，一直下一步直到Master server和所有的media server的备份信息目录都显示出来，按“OK”保存退出；
3）右键点左窗口的“Catalog”选“Configure the Catalog Backup”，出现配置窗口；
“Attributes”栏：“Mdia1”-〉选diak,-〉“Media density”备份目录名如输入/backup/catalog
“Schedule”栏: 选择“After each session of scheduled,user,or manual backups”
按“OK”退出
4．配置策略进行备份
建立oracle在线全备
1）新建一个策略，命名尽量能看出那台主机和备份的概要，如要对gzwebdb进行oracle全备可为：gzwebdb_ora_full；
“Attributes”栏：“Policy”选oracle,->“Policy storage unit”选要备份的主机本地的unit，如gzwebdb-disk；
“Schedules”栏：选“new”按钮->“name”定义一个名称可以策略名相同gzwebdb_ora_full，->“Frequency”选备份的频率，一日一备选
1 Days,->“Retention”选保存的时间，如保存一周选1 week；“start window”->根据需要设置备份策略启动周期的时间，保存按“OK”
；
“Clients”栏：按“new”手工输入备份主机名gzwebdb；
“Backup Selections”栏：按“new”手工输入备份脚本的路径，如/u01/oracle/ora920/backup/back_fulldb.sh
保存退出
2）可对策略进行手工备份，选定设定好策略，按右键选“Manual backup...”，及开始备份。备份过程状态可在窗口的“Activity Monitor
”处监控。
5．oracle备份脚本
在Veritas下设定oracle备份策略开始备份前，相应的oracle脚本需在指定的目录下存在，每个脚本可产生对应的日志文件，便于查看和差错时使用。备份脚本见附件“Veritas_ora_script.rar”，每台备份主机可根据实际情况修改使用。
（1）运行安装程序
# /cdrom1/install
VERITAS Installation Script
Copyright 1993 - 2004 VERITAS Software Corporation, All Rights Reserved.
Installation Options
1 NetBackup
2 NetBackup Client Software
q To quit from this script
Choose an option [default: q]: 1
The NetBackup and Media Manager software is built for use on LINUX hardware.
Do you want to install NetBackup and Media Manager files? [y,n] (y) y
NetBackup and Media Manager are normally installed in /usr/openv.
Is it OK to install in /usr/openv? [y,n] (y) y
The Linux clients will be loaded.
Do you want to load any other NetBackup clients onto the server? [y,n] (y) n
开始安装
Reading client binaries for /usr/openv/netbackup/client/Linux
Linux/
......
+ /bin/cp ./fsanalyze /usr/openv/netbackup/bin/fsanalyze
A NetBackup Server or Enterprise Server license key is needed
for installation to continue.
Enter license key: (输入master licence)
:
NetBackup Enterprise Server Base product with the following features enabled:
Open File Backup
Remote Client Support
Robotic Library Sharing Support
Remote Media Server Support
Microsoft RSM Robotic Libraries
ADIC DAS/SDLC Robotic Libraries
IBM ATL Robotic Libraries
Fujitsu LMF Robotic Libraries
StorageTek ACS Robotic Libraries
Inline Tape Copy
has been registered.
All additional keys should be added at this time.
Do you want to add additional license keys now? [y,n] (y) n
是否增加其他licence，可以选y继续输入或选n等安装完后在管理界面上再添加
If this machine will be using a different network interface than the
default (gzdb1), the name of the preferred interface should be used
as the configured server name. If this machine will be part of a
cluster, the virtual name should be used as the configured server name.
The domainname of your server appears to be "(none)".
You may choose to use this domainname in your configured NetBackup server
name, or simply use "gzdb1" as the configured NetBackup server name.
Would you like to use "gzdb1.(none)" as the configured
name of the NetBackup server? [y,n] (y) n （选n）
Enter the name of the NetBackup server: gzdb1 （手工填写本机名，可不用域名）
Is gzdb1 the master server? [y,n] (y) （问本机是否是master server,选y）
Do you have any media servers? [y,n] (n) （问有无其他media server，暂无选n）
......
安装过程会在/etc/services增加服务
......
To change these entries modify the file /tmp/services.ov_edited.27459
and enter
......
Sending SIGHUP to xinetd process.
Reloading configuration: [ OK ] （系统的xinetd服务必须已启动）
......
安装过程会在/etc/rc.d增加系统启动和关闭服务脚本
......
Enter which host will store the global device database.
(default: gzdb1): （问全局库的主机，如默认显示是master server就按回车，否及手工输入）
Starting the NetBackup database manager process (bpdbm).
Do you want to create policy and schedule examples that you can view or use
when you are configuring your own policies and schedules? [y,n] (y) （问是否创建备份策略调度列表及事例，选y）
......
安装过程在创建policy and schedule
......
Do you want to start the Media Manager device daemon processes? [y,n] (y) （问是否启动Media Manager服务，选y）
Starting the Media Manager device daemon processes.
Do you want to start the NetBackup bprd process so
backups and restores can be initiated? [y,n] (y) （问备份恢复服务是否启动，选y）
Starting the NetBackup request daemon process (bprd).
Done executing NB.inst
VERITAS Installation Script
Copyright 1993 - 2004 VERITAS Software Corporation, All Rights Reserved.
Installation Options
1 NetBackup
2 NetBackup Client Software
q To quit from this script
Choose an option [default: q]: （安装完毕，选q退出）
（2） 编辑.bash_profile
在root目录下的.bash_profile里手工加入Veritas的命令路径，便于日常命令使用。
加入：
PATH=$PATH:$HOME/bin:/usr/openv/netbackup/bin:/usr/openv/netbackup/bin/admincmd:/usr/openv/netbackup/bin/goodies:/usr/openv
/volmgr/bin
保存退出
执行使生效：#. ./.bash_profile
执行bpps -a查看进程，应有以下进程
# bpps -a
NB Processes
------------
root 27633 1 0 11:23 pts/0 00:00:00 /usr/openv/netbackup/bin/bpdbm
root 27906 1 0 11:23 pts/0 00:00:00 /usr/openv/netbackup/bin/bprd
root 27635 27633 0 11:23 pts/0 00:00:00 /usr/openv/netbackup/bin/bpjobd
MM Processes
------------
root 27613 1 0 11:23 pts/0 00:00:00 /usr/openv/volmgr/bin/vmd
（3）编辑/usr/openv/netbackup/bp.conf文件
根据主机脚色加入以下信息
SERVER = gzdb1
SERVER = gzwebdb
SERVER = gzwebdbbak
CLIENT_NAME = gzdb1
MEDIA_SERVER = gzwebdb
MEDIA_SERVER = gzwebdbbak
......
注：SERVER--〉包括master server 和media server；
CLIENT_NAME--〉本机及要通过网络备到本机的主机；
MEDIA_SERVER--〉包括master server 和media server。
（4）安装补丁
安装以下所补丁
NB_51_4_M_280420.linux.tar
NB_CLT_51_4_M_280431.tar
NB_ORA_51_4_M_280463.linux2.4.tar
1）将文件放在一临时目录下，全部用tar接压；
2）用netbackup stop停止netbackup服务；
3）运行安装脚本；
#./Vrts_pack.install
There are 3 packs available in /data/veritas_inst:
(* denotes installed pack)
NB_51_4_M
NB_CLT_51_4_M
NB_ORA_51_4_M
Enter pack name (or q) [q]: NB_51_4_M （在列表里选择第一个补丁）
Pack NB_51_4_M requires the following pack levels(s) :
NB_CLT_51_4_M.
Pack NB_51_4_M will not be installed at this time.
Installing required pack, NB_CLT_51_4_M, now.
Pack NB_CLT_51_4_M co-requires the following pack level(s) :
NB_51_4_M.
Install pack NB_CLT_51_4_M Wed Feb 15 14:56:14 CST 2006 Rev. 1.19.4.23
Performing install of Linux/RedHat2.4 client binaries locally.
Running preinstall script.
......
安装程序会将“NB_CLT_51_4_M”一并安装
......
------------------------------------------------
Installation of pack NB_51_4_M completed Wed Feb 15 14:56:14 CST 2006 Rev. 1.19.4.23.
------------------------------------------------
There are 3 packs available in /data/veritas_inst:
(* denotes installed pack)
NB_51_4_M *
NB_CLT_51_4_M *
NB_ORA_51_4_M
Enter pack name (or q) [q]: （打*号表示已安装的补丁，master server不需安装NB_ORA_51_4_M，选q退出）
4）执行netbackup start重新启动服务。
2．安装media server
（1）运行安装程序
# /cdrom1/install
VERITAS Installation Script
Copyright 1993 - 2004 VERITAS Software Corporation, All Rights Reserved.
Installation Options
1 NetBackup
2 NetBackup Client Software
q To quit from this script
Choose an option [default: q]: 1
The NetBackup and Media Manager software is built for use on LINUX hardware.
Do you want to install NetBackup and Media Manager files? [y,n] (y) y
NetBackup and Media Manager are normally installed in /usr/openv.
Is it OK to install in /usr/openv? [y,n] (y) y
The Linux clients will be loaded.
Do you want to load any other NetBackup clients onto the server? [y,n] (y) n
开始安装
Reading client binaries for /usr/openv/netbackup/client/Linux
Linux/
......
+ /bin/cp ./fsanalyze /usr/openv/netbackup/bin/fsanalyze
A NetBackup Server or Enterprise Server license key is needed
for installation to continue.
Enter license key: (输入master licence)
:
NetBackup Enterprise Server Base product with the following features enabled:
Open File Backup
Remote Client Support
Robotic Library Sharing Support
Remote Media Server Support
Microsoft RSM Robotic Libraries
ADIC DAS/SDLC Robotic Libraries
IBM ATL Robotic Libraries
Fujitsu LMF Robotic Libraries
StorageTek ACS Robotic Libraries
Inline Tape Copy
has been registered.
All additional keys should be added at this time.
Do you want to add additional license keys now? [y,n] (y) n
是否增加其他licence，可以选y继续输入或选n等安装完后在管理界面上再添加
If this machine will be using a different network interface than the
default (gzwebdb), the name of the preferred interface should be used
as the configured server name. If this machine will be part of a
cluster, the virtual name should be used as the configured server name.
The domainname of your server appears to be "(none)".
You may choose to use this domainname in your configured NetBackup server
name, or simply use "gzwebdb" as the configured NetBackup server name.
Would you like to use "gzwebdb.(none)" as the configured
name of the NetBackup server? [y,n] (y) n （选n）
Enter the name of the NetBackup server: gzwebdb （手工填写本机名，可不用域名）
Is gzdb1 the master server? [y,n] (n) （问本机是否是master server,选n）
What is the fully qualified name of the master server? gzdb1 （在此输入master server的主机名，如“gzdb1”）
......
安装过程会在/etc/services增加服务
......
To change these entries modify the file /tmp/services.ov_edited.27459
and enter
......
Sending SIGHUP to xinetd process.
Reloading configuration: [ OK ] （系统的xinetd服务必须已启动）
......
安装过程会在/etc/rc.d增加系统启动和关闭服务脚本
......
Enter global device database host (default: gzdb1): （问全局库的主机，如默认显示是master server就按回车，否及手工输入）
Do you want to start the Media Manager device daemon processes? [y,n] (y) （问是否启动Media Manager服务，选y）
Starting the Media Manager device daemon processes.
Done executing NB.inst
VERITAS Installation Script
Copyright 1993 - 2004 VERITAS Software Corporation, All Rights Reserved.
Installation Options
1 NetBackup
2 NetBackup Client Software
q To quit from this script
Choose an option [default: q]: （安装完毕，选q退出）
（2）编辑.bash_profile
在root目录下的.bash_profile里手工加入Veritas的命令路径，便于日常命令使用。
加入：
PATH=$PATH:$HOME/bin:/usr/openv/netbackup/bin:/usr/openv/netbackup/bin/admincmd:/usr/openv/netbackup/bin/goodies:/usr/openv
/volmgr/bin
保存退出
执行使生效：#. ./.bash_profile
执行bpps -a查看进程，应有以下进程
# bpps -a
NB Processes
------------
MM Processes
------------
root 27613 1 0 11:23 pts/0 00:00:00 /usr/openv/volmgr/bin/vmd
（3）编辑/usr/openv/netbackup/bp.conf文件
根据主机脚色加入以下信息
SERVER = gzdb1
SERVER = gzwebdb
SERVER = gzwebdbbak
CLIENT_NAME = gzdb1
MEDIA_SERVER = gzwebdb
MEDIA_SERVER = gzwebdbbak
......
注：SERVER--〉包括master server 和media server；
CLIENT_NAME--〉本机及要通过网络备到本机的主机；
MEDIA_SERVER--〉包括master server 和media server。
（4）安装oracle agent
因为media server作为client端，需要在线备份oracle数据库，所以要安装database agent用以数据库的备份。
1） 进入加载database agent的光盘或IOS目录
cd /cdrom2
2)运新安装程序
# ./install
VERITAS Installation Script
Copyright 1993 - 2004 VERITAS Software Corporation, All Rights Reserved.
Installation Options
1 NetBackup Add-On Product Software
2 NetBackup Database Agent Software
q To quit from this script
Choose an option [default: q]: 2
**********
There are two ways to install database agent software.
1. Remote Installation: Loads the software on a master server with
the intent of pushing database software out to affected clients.
2. Local Installation: Loads and installs the software only to this
local machine.
**********
Do you want to do a local installation? [y,n] (n) 2
You have entered an invalid option.
Do you want to do a local installation? [y,n] (n) y
**********
NetBackup Database Agent Installation
Choose the Database Agents you wish to install
one at a time or select Install All Database Agents.
1) NetBackup for DB2
2) NetBackup for Informix
3) NetBackup for Lotus Notes
4) NetBackup for Oracle
5) NetBackup for SAP
6) NetBackup for Sybase
7) Install All Database Agents
q) Done Selecting Agents
x) Exit from this Script
Select the Oracle option if upgrading any of the
following: Oracle, Oracle on UNIX Advanced BLI Agent,
Oracle ServerFree Agent or Database Archiver.
The packages have been combined.
Choose an option: 4 （选4，安装oracle类型）
Choose an option: q （如没其他类型的数据库，默认q退出选择状态）
You have chosen to install these Database Agents:
NetBackup for Oracle
Is this list correct? [y,n] (y) （显示当前选择的database 类型，选y开始安装）
**********
Of the agents selected, the following are supported
on this platform and will be installed:
Oracle
Loading the Database Agent packages into the
/usr/openv/netbackup/dbext directory and installing.
**********
Installing NetBackup for Oracle
……安装程序在安装文件
NetBackup for Oracle installation completed.
**********
Execution of ./install_dbext is complete.
Execution of ./install is complete.
File /tmp/install_options_trace.27635 contains a trace of this install.
VERITAS Installation Script
Copyright 1993 - 2004 VERITAS Software Corporation, All Rights Reserved.
Installation Options
1 NetBackup Add-On Product Software
2 NetBackup Database Agent Software
q To quit from this script
Choose an option [default: q]:
退出完成安装
（5）安装补丁
安装以下所补丁
NB_51_4_M_280420.linux.tar
NB_CLT_51_4_M_280431.tar
NB_ORA_51_4_M_280463.linux2.4.tar
1）将文件放在一临时目录下，全部用tar接压；
2）用netbackup stop停止netbackup服务；
3）运行安装脚本；
#./Vrts_pack.install
There are 3 packs available in /data/veritas_inst:
(* denotes installed pack)
NB_51_4_M
NB_CLT_51_4_M
NB_ORA_51_4_M
Enter pack name (or q) [q]: NB_51_4_M （在列表里选择第一个补丁）
Pack NB_51_4_M requires the following pack levels(s) :
NB_CLT_51_4_M.
Pack NB_51_4_M will not be installed at this time.
Installing required pack, NB_CLT_51_4_M, now.
Pack NB_CLT_51_4_M co-requires the following pack level(s) :
NB_51_4_M.
Install pack NB_CLT_51_4_M Wed Feb 15 14:56:14 CST 2006 Rev. 1.19.4.23
Performing install of Linux/RedHat2.4 client binaries locally.
Running preinstall script.
......
安装程序会将“NB_CLT_51_4_M”一并安装
......
------------------------------------------------
Installation of pack NB_51_4_M completed Wed Feb 15 14:56:14 CST 2006 Rev. 1.19.4.23.
------------------------------------------------
There are 3 packs available in /data/veritas_inst:
(* denotes installed pack)
NB_51_4_M *
NB_CLT_51_4_M *
NB_ORA_51_4_M
Enter pack name (or q) [q]: NB_ORA_51_4_M （在列表里选择ORA补丁）
Install pack NB_ORA_51_4_M Wed Feb 15 14:56:14 CST 2006 Rev. 1.19.4.23
Please pick one of the following methods for installing this database pack.
1. Remote Installation: Loads the software on a master server with
the intent of pushing the pack out to affected clients.
2. Local Installation: Loads and installs the pack only to this
local machine.
Enter option (or q) [q]: 2
Saving pre-existing binaries.
Running preinstall script.
......
安装程序会将“NB_ORA_51_4_M”补丁
......
------------------------------------------------
Installation of pack NB_ORA_51_4_M completed Wed Feb 15 15:16:14 CST 2006 Rev. 1.19.4.23.
------------------------------------------------
There are 3 packs available in /data/veritas_inst:
(* denotes installed pack)
NB_51_4_M *
NB_CLT_51_4_M *
NB_ORA_51_4_M *
Enter pack name (or q) [q]: （打*号表示已安装的补丁，选q退出）
4）执行netbackup start重新启动服务。
5）用Veritas链接数据库
Veritas必须和数据库进行链接后才可以正常使用，使用veritas自带脚本（位置/usr/openv/netbackup/bin/oracle_link）进行数据库链接
，直接运行即可：
/usr/openv/netbackup/bin/oracle_link
三、配置Veritas
1．图形管理界面调用
1）用图形工具，如VNC打开master server图形界面，新建一个终端窗口。
2）运行/usr/openv/netbackup/bin/jnbSA命令出现“Netbackup Admin Console”界面。
2．配置Storage Units
在“Storage Units”处配置master server和media server的备份设备。
master server:
例（主机名为gzdb1，备份目录为/backup）
按右键选new
"Storage unit name"输入gzdb1-disk
"NerBackup media server"输入gzdb1
"Storage unit tupe"选disk
"Absolute pathname to directory"输入/backup
按"OK"退出
media server:
例（主机名为gzwebdb，备份目录为/backup）
按右键选new
"Storage unit name"输入gzwebdb-disk
"NerBackup media server"输入gzwebdb
"Storage unit tupe"选disk
"Absolute pathname to directory"输入/backup
按"OK"退出
......
如此将所有media server的主机都新建起来。
3．配置Veritas的catalog信息库备份
1）点左窗口的最上方的“gzdb1(Master Server)”->再选右边的“Configure the Catalog Backup”配置向导；
2）按提示操作选择，一直下一步直到Master server和所有的media server的备份信息目录都显示出来，按“OK”保存退出；
3）右键点左窗口的“Catalog”选“Configure the Catalog Backup”，出现配置窗口；
“Attributes”栏：“Mdia1”-〉选diak,-〉“Media density”备份目录名如输入/backup/catalog
“Schedule”栏: 选择“After each session of scheduled,user,or manual backups”
按“OK”退出
4．配置策略进行备份
建立oracle在线全备
1）新建一个策略，命名尽量能看出那台主机和备份的概要，如要对gzwebdb进行oracle全备可为：gzwebdb_ora_full；
“Attributes”栏：“Policy”选oracle,->“Policy storage unit”选要备份的主机本地的unit，如gzwebdb-disk；
“Schedules”栏：选“new”按钮->“name”定义一个名称可以策略名相同gzwebdb_ora_full，->“Frequency”选备份的频率，一日一备选
1 Days,->“Retention”选保存的时间，如保存一周选1 week；“start window”->根据需要设置备份策略启动周期的时间，保存按“OK”
；
“Clients”栏：按“new”手工输入备份主机名gzwebdb；
“Backup Selections”栏：按“new”手工输入备份脚本的路径，如/u01/oracle/ora920/backup/back_fulldb.sh
保存退出
2）可对策略进行手工备份，选定设定好策略，按右键选“Manual backup...”，及开始备份。备份过程状态可在窗口的“Activity Monitor
”处监控。
5．oracle备份脚本
在Veritas下设定oracle备份策略开始备份前，相应的oracle脚本需在指定的目录下存在，每个脚本可产生对应的日志文件，便于查看和差错时使用。备份脚本见附件“Veritas_ora_script.rar”，每台备份主机可根据实际情况修改使用。
Redhat下MD设备的多路径使用问题
使用MD设备；创建文件系统之后，有时候在I/O操作的时候（主要是写），会出现I/O Error，然后文件系统被remount成ReadOnly，该问题在测试及系统上线的时候多次出现，经检查，是在创建MD虚拟设备之前创建了文件系统，下面详细记录了问题出现、处理的过程，及详细的原因。
一、错误重现
以下步骤与3月11日系统迁移中的过程一致，在6920上新映射了一个5G的LUN到主机，然后重复建立MD设备、创建文件系统及cp文件的过程，最后dmesg得出了与当天相同的日志。
1、对LUN进行分区（此前已经扫描出sd设备）：
[root@gddb2 ~]# ./getwwn.sh
3600015d00004d30000000000000007e5 /dev/sda
3600015d00004d30000000000000007e5 /dev/sdd
3600015d00004d300000000000000081e /dev/sdb
3600015d00004d300000000000000081e /dev/sde
3600015d000055c000000000000000a18 /dev/sdc
3600015d000055c000000000000000a18 /dev/sdf
[root@gddb2 ~]# fdisk -l /dev/sdb
Disk /dev/sdb: 5368 MB, 5368709120 bytes
166 heads, 62 sectors/track, 1018 cylinders
Units = cylinders of 10292 * 512 = 5269504 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 1018 5238597 83 Linux
[root@gddb2 ~]# fdisk -l /dev/sde
Disk /dev/sde: 5368 MB, 5368709120 bytes
166 heads, 62 sectors/track, 1018 cylinders
Units = cylinders of 10292 * 512 = 5269504 bytes
Device Boot Start End Blocks Id System
/dev/sde1 1 1018 5238597 83 Linux
――――由于sdb和sde是同一个LUN的不同路径，所以对sdb进行分区就可以了
2、在sdb1上创建ext3文件系统
[root@gddb2 ~]# mkfs.ext3 /dev/sdb1
mke2fs 1.35 (28-Feb-2004)
max_blocks 1341080576, rsv_groups = 40927, rsv_gdb = 319
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
655360 inodes, 1309649 blocks
65482 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1342177280
40 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736
Writing inode tables: done
inode.i_blocks = 20424, i_size = 4243456
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 37 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
3、创建MD虚拟设备：
[root@gddb2 ~]# mdadm -C /dev/md3 -l multipath -n 2 /dev/sdb1 /dev/sde1
mdadm: /dev/sdb1 appears to contain an ext2fs file system
size=5238596K mtime=Thu Jan 1 08:00:00 1970
mdadm: /dev/sde1 appears to contain an ext2fs file system
size=5238596K mtime=Thu Jan 1 08:00:00 1970
Continue creating array? y
mdadm: array /dev/md3 started.
4、加载文件系统，并做cp操作：
[root@gddb2 ~]# mount /dev/md3 /md3/
[root@gddb2 ~]# cp /data/tablespace/* /md3/
cp: writing `/md3/mtamail21.dbf': No space left on device
cp: writing `/md3/mtamail22.dbf': No space left on device
cp: writing `/md3/mtamail23.dbf': No space left on device
cp: writing `/md3/mtamail31.dbf': No space left on device
cp: writing `/md3/mtamail32.dbf': No space left on device
cp: writing `/md3/mtamail33.dbf': No space left on device
cp: writing `/md3/mtamanager01.dbf': No space left on device
5、以上的cp命令只是报了文件系统空间满的错，但是没有报I/O错，再看看系统的日志dmesg：
[root@gddb2 ~]# dmesg
……（省略若干无关内容）
md: export_rdev(sdb1)
md: bind
md: bind
multipath: array md3 active with 2 out of 2 IO paths
kjournald starting. Commit interval 5 seconds
EXT3 FS on md3, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
SELinux: initialized (dev md3, type ext3), uses xattr
attempt to access beyond end of device
md3: rw=1, want=10477064, limit=10477056
printk: 7 messages suppressed.
Buffer I/O error on device md3, logical block 1309632
lost page write due to I/O error on md3
attempt to access beyond end of device
md3: rw=1, want=10477072, limit=10477056
Buffer I/O error on device md3, logical block 1309633
lost page write due to I/O error on md3
attempt to access beyond end of device
md3: rw=1, want=10477080, limit=10477056
Buffer I/O error on device md3, logical block 1309634
lost page write due to I/O error on md3
attempt to access beyond end of device
md3: rw=1, want=10477088, limit=10477056
Buffer I/O error on device md3, logical block 1309635
lost page write due to I/O error on md3
attempt to access beyond end of device
md3: rw=1, want=10477096, limit=10477056
Buffer I/O error on device md3, logical block 1309636
lost page write due to I/O error on md3
attempt to access beyond end of device
md3: rw=1, want=10477104, limit=10477056
Buffer I/O error on device md3, logical block 1309637
lost page write due to I/O error on md3
attempt to access beyond end of device
md3: rw=1, want=10477112, limit=10477056
Buffer I/O error on device md3, logical block 1309638
lost page write due to I/O error on md3
attempt to access beyond end of device
md3: rw=1, want=10477120, limit=10477056
Buffer I/O error on device md3, logical block 1309639
lost page write due to I/O error on md3
attempt to access beyond end of device
md3: rw=1, want=10477128, limit=10477056
Buffer I/O error on device md3, logical block 1309640
lost page write due to I/O error on md3
attempt to access beyond end of device
md3: rw=1, want=10477136, limit=10477056
Buffer I/O error on device md3, logical block 1309641
lost page write due to I/O error on md3
attempt to access beyond end of device
md3: rw=1, want=10477144, limit=10477056
attempt to access beyond end of device
md3: rw=1, want=10477152, limit=10477056
attempt to access beyond end of device
md3: rw=1, want=10477160, limit=10477056
attempt to access beyond end of device
md3: rw=1, want=10477168, limit=10477056
attempt to access beyond end of device
md3: rw=1, want=10477176, limit=10477056
attempt to access beyond end of device
md3: rw=1, want=10477184, limit=10477056
attempt to access beyond end of device
md3: rw=1, want=10477192, limit=10477056
可见与3月11日系统迁移的时候报错一致。
二、错误原因分析
这一次主要的报错信息意义分析：
attempt to access beyond end of device
md1: rw=1, want=419424776, limit=419424768
Buffer I/O error on device md1, logical block 52428096
大致意思是：尝试访问超过设备结尾的块，第3行提示访问的逻辑块为52428096。
而md1设备的最大逻辑块是52428119：
[root@gddb2 tablespace]# dumpe2fs /dev/md1
dumpe2fs 1.35 (28-Feb-2004)
Filesystem volume name:
Last mounted on:
Filesystem UUID: 76ee30b6-aac9-484a-8b76-5b76a7c31253
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr resize_inode filetype needs_recovery sparse_super large_file
Default mount options: (none)
Filesystem state: clean with errors
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 26214400
Block count: 52428119
Reserved block count: 2621405
Free blocks: 38174314
Free inodes: 26214307
First block: 0
Block size: 4096
Fragment size: 4096
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 16384
Inode blocks per group: 512
Filesystem created: Wed Mar 1 18:34:53 2006
Last mount time: Mon Mar 13 16:48:43 2006
Last write time: Mon Mar 13 16:48:43 2006
Mount count: 8
Maximum mount count: 32
Last checked: Wed Mar 1 18:34:53 2006
Check interval: 15552000 (6 months)
Next check after: Mon Aug 28 18:34:53 2006
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 128
Journal inode: 8
Default directory hash: tea
Directory Hash Seed: f96d19fa-fb5c-43e4-a6d7-410470ecfc0d
Journal backup: inode blocks
可见，是在临近文件系统结尾的20多个逻辑块出了问题。而实际上这20多个逻辑块
md1设备上是不存在的；原因是先创建ext3文件系统，而没有给md设备预留superblock空间（md设备需要在物理设备尾部部分空间保存md multipath信息，见附）；在创建md设备之后，物理设备的尾部分被md的superblock占用了几k到几M的空间（通常为64K），但文件系统并不知情，仍然按原来（创建md设备前）空间结构填写inode的bitmap；inode指针错位，导致最后几个逻辑块“悬空”，写的时候就会报类似“attempt to access beyond end of device”的错误。
以第一节里面的5G的LUN为例，如果在创建了md设备之后再创建ext3文件系统，则文件系统的逻辑块会比原来（创建ext3文件系统后创建md设备）少若干（几十个）：
[root@gddb2 ~]# umount /md3/
[root@gddb2 ~]# dumpe2fs /dev/sdb1>sdb1
[root@gddb2 ~]# mkfs.ext3 /dev/md3
mke2fs 1.35 (28-Feb-2004)
max_blocks 1341063168, rsv_groups = 40926, rsv_gdb = 319
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
655360 inodes, 1309632 blocks
65481 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1342177280
40 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736
Writing inode tables: done
inode.i_blocks = 20424, i_size = 4243456
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 28 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
[root@gddb2 ~]# dumpe2fs >md3
dumpe2fs 1.35 (28-Feb-2004)
Usage: dumpe2fs [-bfhixV] [-ob superblock] [-oB blocksize] device
[root@gddb2 ~]# dumpe2fs /dev/md3>md3
dumpe2fs 1.35 (28-Feb-2004)
[root@gddb2 ~]# diff md3 sdb1more
3c3
<> Filesystem UUID: 48a10ffb-d7ef-41d5-9ef6-a277f0c191f3
12,14c12,14
<> Block count: 1309649
> Reserved block count: 65482
> Free blocks: 1278313
23c23
<> Filesystem created: Mon Mar 13 18:07:59 2006
25c25
<> Last write time: Mon Mar 13 18:07:59 2006
27,28c27,28
<> Maximum mount count: 37
> Last checked: Mon Mar 13 18:07:59 2006
30c30
<> Next check after: Sat Sep 9 18:07:59 2006
37c37
<> Directory Hash Seed: 51a94659-8e0d-468d-955c-b239aef0abff
283c283
< Group 39: (Blocks 1277952-1309631)
对比可见，两种方式下创建的文件系统刚好差了1309649-1309632=17个逻辑块；由于md会保留最后的64K空间作为superblock，md3的blocksize是4096（4K），被占用的逻辑块就为：64K/4K=16个，与上面的结果相似；而在第一部分的测试中，开始报错的逻辑块刚好是1309632 （Buffer I/O error on device md3, logical block 1309632），印证了创建了md虚拟设备后的实际块设备大小（1309632个逻辑块）。
三、解决方法及建议：
在对LUN进行分区、创建md虚拟设备之后创建ext3文件系统。
一、错误重现
以下步骤与3月11日系统迁移中的过程一致，在6920上新映射了一个5G的LUN到主机，然后重复建立MD设备、创建文件系统及cp文件的过程，最后dmesg得出了与当天相同的日志。
1、对LUN进行分区（此前已经扫描出sd设备）：
[root@gddb2 ~]# ./getwwn.sh
3600015d00004d30000000000000007e5 /dev/sda
3600015d00004d30000000000000007e5 /dev/sdd
3600015d00004d300000000000000081e /dev/sdb
3600015d00004d300000000000000081e /dev/sde
3600015d000055c000000000000000a18 /dev/sdc
3600015d000055c000000000000000a18 /dev/sdf
[root@gddb2 ~]# fdisk -l /dev/sdb
Disk /dev/sdb: 5368 MB, 5368709120 bytes
166 heads, 62 sectors/track, 1018 cylinders
Units = cylinders of 10292 * 512 = 5269504 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 1018 5238597 83 Linux
[root@gddb2 ~]# fdisk -l /dev/sde
Disk /dev/sde: 5368 MB, 5368709120 bytes
166 heads, 62 sectors/track, 1018 cylinders
Units = cylinders of 10292 * 512 = 5269504 bytes
Device Boot Start End Blocks Id System
/dev/sde1 1 1018 5238597 83 Linux
――――由于sdb和sde是同一个LUN的不同路径，所以对sdb进行分区就可以了
2、在sdb1上创建ext3文件系统
[root@gddb2 ~]# mkfs.ext3 /dev/sdb1
mke2fs 1.35 (28-Feb-2004)
max_blocks 1341080576, rsv_groups = 40927, rsv_gdb = 319
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
655360 inodes, 1309649 blocks
65482 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1342177280
40 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736
Writing inode tables: done
inode.i_blocks = 20424, i_size = 4243456
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 37 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
3、创建MD虚拟设备：
[root@gddb2 ~]# mdadm -C /dev/md3 -l multipath -n 2 /dev/sdb1 /dev/sde1
mdadm: /dev/sdb1 appears to contain an ext2fs file system
size=5238596K mtime=Thu Jan 1 08:00:00 1970
mdadm: /dev/sde1 appears to contain an ext2fs file system
size=5238596K mtime=Thu Jan 1 08:00:00 1970
Continue creating array? y
mdadm: array /dev/md3 started.
4、加载文件系统，并做cp操作：
[root@gddb2 ~]# mount /dev/md3 /md3/
[root@gddb2 ~]# cp /data/tablespace/* /md3/
cp: writing `/md3/mtamail21.dbf': No space left on device
cp: writing `/md3/mtamail22.dbf': No space left on device
cp: writing `/md3/mtamail23.dbf': No space left on device
cp: writing `/md3/mtamail31.dbf': No space left on device
cp: writing `/md3/mtamail32.dbf': No space left on device
cp: writing `/md3/mtamail33.dbf': No space left on device
cp: writing `/md3/mtamanager01.dbf': No space left on device
5、以上的cp命令只是报了文件系统空间满的错，但是没有报I/O错，再看看系统的日志dmesg：
[root@gddb2 ~]# dmesg
……（省略若干无关内容）
md: export_rdev(sdb1)
md: bind
md: bind
multipath: array md3 active with 2 out of 2 IO paths
kjournald starting. Commit interval 5 seconds
EXT3 FS on md3, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
SELinux: initialized (dev md3, type ext3), uses xattr
attempt to access beyond end of device
md3: rw=1, want=10477064, limit=10477056
printk: 7 messages suppressed.
Buffer I/O error on device md3, logical block 1309632
lost page write due to I/O error on md3
attempt to access beyond end of device
md3: rw=1, want=10477072, limit=10477056
Buffer I/O error on device md3, logical block 1309633
lost page write due to I/O error on md3
attempt to access beyond end of device
md3: rw=1, want=10477080, limit=10477056
Buffer I/O error on device md3, logical block 1309634
lost page write due to I/O error on md3
attempt to access beyond end of device
md3: rw=1, want=10477088, limit=10477056
Buffer I/O error on device md3, logical block 1309635
lost page write due to I/O error on md3
attempt to access beyond end of device
md3: rw=1, want=10477096, limit=10477056
Buffer I/O error on device md3, logical block 1309636
lost page write due to I/O error on md3
attempt to access beyond end of device
md3: rw=1, want=10477104, limit=10477056
Buffer I/O error on device md3, logical block 1309637
lost page write due to I/O error on md3
attempt to access beyond end of device
md3: rw=1, want=10477112, limit=10477056
Buffer I/O error on device md3, logical block 1309638
lost page write due to I/O error on md3
attempt to access beyond end of device
md3: rw=1, want=10477120, limit=10477056
Buffer I/O error on device md3, logical block 1309639
lost page write due to I/O error on md3
attempt to access beyond end of device
md3: rw=1, want=10477128, limit=10477056
Buffer I/O error on device md3, logical block 1309640
lost page write due to I/O error on md3
attempt to access beyond end of device
md3: rw=1, want=10477136, limit=10477056
Buffer I/O error on device md3, logical block 1309641
lost page write due to I/O error on md3
attempt to access beyond end of device
md3: rw=1, want=10477144, limit=10477056
attempt to access beyond end of device
md3: rw=1, want=10477152, limit=10477056
attempt to access beyond end of device
md3: rw=1, want=10477160, limit=10477056
attempt to access beyond end of device
md3: rw=1, want=10477168, limit=10477056
attempt to access beyond end of device
md3: rw=1, want=10477176, limit=10477056
attempt to access beyond end of device
md3: rw=1, want=10477184, limit=10477056
attempt to access beyond end of device
md3: rw=1, want=10477192, limit=10477056
可见与3月11日系统迁移的时候报错一致。
二、错误原因分析
这一次主要的报错信息意义分析：
attempt to access beyond end of device
md1: rw=1, want=419424776, limit=419424768
Buffer I/O error on device md1, logical block 52428096
大致意思是：尝试访问超过设备结尾的块，第3行提示访问的逻辑块为52428096。
而md1设备的最大逻辑块是52428119：
[root@gddb2 tablespace]# dumpe2fs /dev/md1
dumpe2fs 1.35 (28-Feb-2004)
Filesystem volume name:
Last mounted on:
Filesystem UUID: 76ee30b6-aac9-484a-8b76-5b76a7c31253
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr resize_inode filetype needs_recovery sparse_super large_file
Default mount options: (none)
Filesystem state: clean with errors
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 26214400
Block count: 52428119
Reserved block count: 2621405
Free blocks: 38174314
Free inodes: 26214307
First block: 0
Block size: 4096
Fragment size: 4096
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 16384
Inode blocks per group: 512
Filesystem created: Wed Mar 1 18:34:53 2006
Last mount time: Mon Mar 13 16:48:43 2006
Last write time: Mon Mar 13 16:48:43 2006
Mount count: 8
Maximum mount count: 32
Last checked: Wed Mar 1 18:34:53 2006
Check interval: 15552000 (6 months)
Next check after: Mon Aug 28 18:34:53 2006
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 128
Journal inode: 8
Default directory hash: tea
Directory Hash Seed: f96d19fa-fb5c-43e4-a6d7-410470ecfc0d
Journal backup: inode blocks
可见，是在临近文件系统结尾的20多个逻辑块出了问题。而实际上这20多个逻辑块
md1设备上是不存在的；原因是先创建ext3文件系统，而没有给md设备预留superblock空间（md设备需要在物理设备尾部部分空间保存md multipath信息，见附）；在创建md设备之后，物理设备的尾部分被md的superblock占用了几k到几M的空间（通常为64K），但文件系统并不知情，仍然按原来（创建md设备前）空间结构填写inode的bitmap；inode指针错位，导致最后几个逻辑块“悬空”，写的时候就会报类似“attempt to access beyond end of device”的错误。
以第一节里面的5G的LUN为例，如果在创建了md设备之后再创建ext3文件系统，则文件系统的逻辑块会比原来（创建ext3文件系统后创建md设备）少若干（几十个）：
[root@gddb2 ~]# umount /md3/
[root@gddb2 ~]# dumpe2fs /dev/sdb1>sdb1
[root@gddb2 ~]# mkfs.ext3 /dev/md3
mke2fs 1.35 (28-Feb-2004)
max_blocks 1341063168, rsv_groups = 40926, rsv_gdb = 319
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
655360 inodes, 1309632 blocks
65481 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1342177280
40 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736
Writing inode tables: done
inode.i_blocks = 20424, i_size = 4243456
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 28 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
[root@gddb2 ~]# dumpe2fs >md3
dumpe2fs 1.35 (28-Feb-2004)
Usage: dumpe2fs [-bfhixV] [-ob superblock] [-oB blocksize] device
[root@gddb2 ~]# dumpe2fs /dev/md3>md3
dumpe2fs 1.35 (28-Feb-2004)
[root@gddb2 ~]# diff md3 sdb1more
3c3
<> Filesystem UUID: 48a10ffb-d7ef-41d5-9ef6-a277f0c191f3
12,14c12,14
<> Block count: 1309649
> Reserved block count: 65482
> Free blocks: 1278313
23c23
<> Filesystem created: Mon Mar 13 18:07:59 2006
25c25
<> Last write time: Mon Mar 13 18:07:59 2006
27,28c27,28
<> Maximum mount count: 37
> Last checked: Mon Mar 13 18:07:59 2006
30c30
<> Next check after: Sat Sep 9 18:07:59 2006
37c37
<> Directory Hash Seed: 51a94659-8e0d-468d-955c-b239aef0abff
283c283
< Group 39: (Blocks 1277952-1309631)
对比可见，两种方式下创建的文件系统刚好差了1309649-1309632=17个逻辑块；由于md会保留最后的64K空间作为superblock，md3的blocksize是4096（4K），被占用的逻辑块就为：64K/4K=16个，与上面的结果相似；而在第一部分的测试中，开始报错的逻辑块刚好是1309632 （Buffer I/O error on device md3, logical block 1309632），印证了创建了md虚拟设备后的实际块设备大小（1309632个逻辑块）。
三、解决方法及建议：
在对LUN进行分区、创建md虚拟设备之后创建ext3文件系统。
订阅：
评论 (Atom)