bash parse line by line

尋日寫個script去一行行咁process一個list（你可以當係ls 嘅result），上google search "bash parse line by line"搵極都搵唔到個啱我用嘅example．以我嘅性格，通常明知係得嘅我會堅持，亦即係正所謂嘅死硬派，今次我就選擇錯試（即係try an error）．以我所知，bash會split object based on IFS，即係Input File Separator．於是我就寫左一個script如下：－

OLDIFS=${IFS}
IFS=\n
for line in `ls` ; do
echo "Line read is : ${line}"
done
IFS=${OLDIFS}

但係個output就唔太似樣：－

Line read is :-rw-r--r-- 1 root other 2797 Oct 21 2003 file1.txt　-rw-r--r-- 1 root other 32797 Oct 22 2003 file2.txt　．．．

一睇返ls嘅result：－
ls
-rw-r--r-- 1 root other 2797 Oct 21 2003 file1.txt
-rw-r--r-- 1 root other 32797 Oct 22 2003 file2.txt

就知原來佢仲唔識睇\n，以為係"\"同"n"都係一個separator，之後當然試下用single quote, double quote, xx quote....去quote住個\n，但係result當然係一樣啦．．．．真係比佢激死．．．玩足成日都無咩inspiration，於是就決定明日愁來明日當，立刻打道回府，照顧BB去也．今朝返到公司，突然就比我醒起，原來自己N個月前正好寫左個script，又係要line by line咁parse個file．所以都話，有時當你諗唔起點做嘅時候，最好都係唔好開OT，即刻返屋企hea下就反而會有新靈感．亦即係唔係比個死胡同困住，退一步，諗一諗，唔得嘅話就再停一停，先再諗一諗．．．哈哈．．．一睇之下，就發現原來自己太大意，其實只要睇下Advanced Bash Scripting Guide嘅Chapter 5 (Quoting)，就會知道要OS真正了解\n=newline，就要用$加single quote．．．之後只係改左少少如下：－


OLDIFS=${IFS}
IFS=$'\n'
for line in `ls` ; do
echo "Line read is : ${line}"
done
IFS=${OLDIFS}

就成功左．．．．真係有時最基本嘅嘢就最唔記得，比自己玩死．．．

mailgraph on RHEL 3

為免mon住第二部mail server，我其實之前已經install過一次mail graph，但係好神奇咁成個mailgraph唔見左．．．

Anyway，可能自己之前發神經install錯左响第二部server上面啦，其實而家真係講咩都無意思，嘅然無左就再install過咪算囉．．．其實mailgraph嘅install都幾易，只係上去佢個website download個tar.gz落嚟．然後跟住佢個README做就得．不過世事點會咁易就搞得掂架．．．

為左唔駛下次又R哂頭都唔知點算，我就將今次嘅經驗寫低落嚟：－

cd tmp
mkdir mailgraph
cd mailgraph
wget http://mailgraph.schweikert.ch/pub/mailgraph-1.14.tar.gz

tar -xvzf mailgraph-1.14.tar.gz
cd mailgraph-1.14

cp mailgraph.cgi /var/www/cgi-bin/
cp mailgraph.pl /usr/local/bin
cp maiilgraph-init /etc/init.d

chkconfig --add mailgraph-init

本來以為已經搞掂，但係當然無咁易啦，一行"server maiilgraph-init start"就比佢鬧”mailgraph-init: unrecognized service”．．．但係我”chkconfig --list”又見”mailgraph-init 0:off 1:off 2:on 3:on 4:on 5:on 6:off”，之後試吓"chkconfig --del mailgraph-init"再用"chkconfig --add mailgraph-init" add佢一次，但係個result都係一樣．check左/etc/init.d/mailgraph-init响度，不過一睇"ls /etc/init.d"就發覺有d唔對路，點解個"mailgraph-init"咁特別唔同色嘅？哈哈，原來無有execute flag，行完"chmod +x /etc/init.d/mailgraph-init"之後當然搞掂啦，其實加多一句响個help度啫，都咁難？定expect人地一定知道呢？不過我估佢係要我地一定要清楚個permission setting，好事嚟嘅，不過都可以加一句架．．．．唉！

Anyway，事情當然無咁快就完結啦，一行”service mailgraph-init start”佢就鬧：－

Starting mail statistics grapher: mailgraphCan't locate RRDs.pm in @INC (@INC contains: /usr/lib/perl5/5.8.0/i386-linux-thread-multi /usr/lib/perl5/5.8.0 /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.0 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.0 /usr/lib/perl5/vendor_perl /usr/lib/perl5/5.8.0/i386-linux-thread-multi /usr/lib/perl5/5.8.0 .) at /usr/local/bin/mailgraph.pl line 360.BEGIN failed--compilation aborted at /usr/local/bin/mailgraph.pl line 360.

但係我ls /usr/local就見有個rrdtool-1.0.49 install左，不過當然去睇/usr/lib/perl5就搵極都唔見果d "RRDx.pm" (x for 好多個咁解），又唔想自己copy過去, source又唔見左，於是咪去http://oss.oetiker.ch/諗住download latest stable嚟install囉，本來佢都有一份好詳細嘅documentation比我跟住嚟install，但係當install到最last一個step嘅時候就比佢鬧：－

/usr/bin/ld: cannot find -lpngcollect2: ld returned 1 exit status

有無搞錯啊．．．search左一陣internet都搵唔到咩頭緒，於是咪上去佢個website download返個舊version (1.0.50)嚟install，諗住實得添．Anyway，上網search到個relevant嘅installation help，不過當然無咁易放過我啦，當我行：－

perl -MCPAN -e shell
install Time::HiRes

佢就鬧我：－

NOTE: if you get an error like this (the Makefile line number may vary):Makefile:91: *** missing separatorthen set the environment variable LC_ALL to "C" and retryfrom scratch (re-run perl "Makefile.PL").(And consider upgrading your Perl to, say, at least Perl 5.8.8.)(You got this message because you seem to have an UTF-8 locale active in your shell environment, this used to cause broken Makefiles to be created from Makefile.PLs)Makefile:91: *** missing separator. Stop. /usr/bin/make -- NOT OKRunning make test Can't test without successful makeRunning make install make had returned bad status, install seems impossible

吓？係唔係玩咁咁盡啊？唉，又唔識點manual install，又唔想update去5.8.8，費事行梗嘅嘢唔work都唔知吖嗎．．．試過加"force install Time::HiRes"都係一樣．好彩，search左一陣就比我搵到有人教我點manual install．於是我就跟住佢教咁：－

wget http://search.cpan.org/CPAN/authors/id/J/JH/JHI/Time-HiRes-1.9715.tar.gz
tar -xvzf Time-HiRes-1.9715.tar.gz
cd Time-HiRes-1.9715.tar.gz
perl Makefile.PL

點知都未make就比佢鬧

NOTE: if you get an error like this (the Makefile line number may vary):
Makefile:91: *** missing separator
then set the environment variable LC_ALL to "C" and retry
from scratch (re-run perl "Makefile.PL").
(And consider upgrading your Perl to, say, at least Perl 5.8.8.)
(You got this message because you seem to have
an UTF-8 locale active in your shell environment, this used
to cause broken Makefiles to be created from Makefile.PLs)
[root@mmsmtp Time-HiRes-1.9715]# make
Makefile:91: *** missing separator. Stop.

但係今次清楚要做咩：－

LC_ALL="C"
export LC_ALL

跟住都係真係跟住佢嘅instruction：－

perl Makefile.PL
make
make test
make install

然後就咁install File::Tail

perl -MCPAN -e shell
install File::Tail
quit

YEAH，點不知一行佢就無嘢睇，所有圖都corrupt哂，check過"/var/logl/httpd/access_log”同"/var/log/httpd/error_log"都無咩meaningful嘅message．好彩我有第二部server行梗，咁就梗係對吓兩邊嘅mailgraph.cgi睇吓有咩唔妥，點不知一睇就見比兩行錯path陰到添．

my $rrd = 'mailgraph.rrd'; # path to where the RRD database is
my $rrd_virus = 'mailgraph_virus.rrd'; # path to where the Virus RRD database is

大佬，要改你就話聲我聽咪得囉，唔駛咁陰濕架．．．搵到果d rrd database原來係一樣都係响/var/lib，於是就改果兩句，用返條啱嘅path．

my $rrd = '/var/lib/mailgraph.rrd'; # path to where the RRD database is
my $rrd_virus = '/var/lib/mailgraph_virus.rrd'; # path to where the Virus RRD database is

之後點？梗係work啦，唔好玩我啦．．．

postfix原來仲未好返。。。

上次update完個perl之後，個amavisd唔work，搞左成日終於都work返，之後又發覺個postfix仲係傻吓傻吓，佢成日話"Apr 28 11:30:09 gateway postfix/smtpd[22203]: NOQUEUE: reject: RCPT from abc.cde.com[108.118.128.138]: 450 4.1.1 <info@mydomain.com>: Recipient address rejected: User unknown in local recipient table; from=<user@cde.com> to=<info@mydomain.com> proto=ESMTP helo="．．．．有無搞錯啊？？？User unknown in local recipient table?個aliases明明有呢個entry喎。。。

唉。。。睇吓個aliases同個aliases.db嘅timestamp又match喎，無理由，試吓reload佢睇吓點？點知一樣。去盡d咪又試吓restart都唔得，再試埋stop之後再restart都唔得。。。

真係有無搞錯，之後唯有試吓postalias hash:aliases用newaliases啦，點知真係work喎。。。無啦啦同個aliases斷左link?有無搞錯啊。。。真係比佢玩死。。。

諗諗吓，好似上次行yum update，佢update左個postfix，原本已經backup哂d config，update完即刻check都一樣，restart左個postfix諗住無咩事添，點知出左事都唔知，所以話don't fix when it aint break係真係至理明言。。。

Compress::Zlib 無啦啦corrupt左

今日比人投訴，話成個月都收唔到email．．．入去mail server度睇，初初就以為咁啱無人send比佢啫，因為睇d log都似正常，懶懶地咪用個gmail send張嚟試吓，點知等左成個鐘都收唔到添．．．於是就再慢慢比多d心機睇吓個maillog，點知見到”mailgateway postfix/lmtp[17734]: connect to 127.0.0.1[127.0.0.1]: Connection refused (port 10024)”吓？？？10024咪就係amavisd囉．．．跟住仲發現個process真係無左添．．．

明明行左半年都無事，又會發啦啦嘅．．．諗吓諗吓，又真係喎，點解近排我無收到worm嘅通知呢？（因為config到一收到有worm嘅email就通知我，雖然我都無嘢做到，不過得個知字都好啊．．．）於是就梗係試吓start返個process啦，點不知又比佢鬧鬼我：

Starting amavisd: ERROR: MISSING REQUIRED BASIC MODULES: Compress::ZlibBEGIN failed--compilation aborted at /usr/local/sbin/amavisd line 160. [FAILED]
我梗係睇吓有無install到啦，做左個"yum list all grep perl"
perl-Compress-Zlib.i386 1.42-1.fc6 installed
吓？咁即係點啊？
我先remove "yum remove perl-Compress-Zlib.i386"之後再install，結果一樣．．．

又係唔理性嘅時候，唔理佢咁多再用"perl -MCPAN -e shell"之後行"install Compress::Zlib"梗係fail啦，到左咁唔理性嘅時候就梗係來硬嚟啦，今次仲”force install Compress::Zlib"．之後梗係install ok架，不過個case都係一樣囉．．．．唔理性到極點就去試吓download最新嘅amavis stable version 2.5.4試吓啦：－

Starting amavisd: ERROR: MISSING REQUIRED BASIC MODULES: Compress::ZlibBEGIN failed--compilation aborted at /usr/local/sbin/amavisd line 171. [FAILED]

呢個做法，只係將個complain嘅line no 由160變左做171．．．即係問題仲”未解決”到啦．．．

R哂頭，明明响度架，又比我搵到有人教’perl -MCompress::Zlib -e'print "Found\n"'’哈哈，行完終於都有少少頭緒喇，因為佢鬧：－

is only avaliable with the XS version at /usr/lib/perl5/site_perl/5.8.8/Compress/Zlib.pm line 9BEGIN failed--compilation aborted at /usr/lib/perl5/site_perl/5.8.8/Compress/Zlib.pm line 9.Compilation failed in require.BEGIN failed--compilation aborted.
yeah，睇嚟要做一次recompile，於是就走入CPAN個shell做一次recompile，即係”perl -MCPAN -e shell"之後打"recompile"．等左半個鐘左右，哈哈，真係work喎！！Yeah．．．但係唔明點解無啦啦d library會corrupt左，不過真係比佢玩死．．．

諗吓諗吓，我好似行過”yum update”去update部機喎，又好似有d perl library update左喎．．．如果係唔得嘅話，仲咩比我update啊？？？

ArcServe Installation Part 2

上回講到installation script行完左，不過究竟點先行到個GUI呢？

其實佢個GUI係一個web interface(如果network down左又start唔到個x咪等死？)．先用cstatus睇吓係唔係所有service都start哂．

[root@server sbin]# ./cstatus
caservd 20020
cadiscovd 20022 20021
cauthd 20024
caloggerd 20025
cadbd 20026
camediad 20127 20126 20027
caqd 20028
staging 20139
cprocess

cacommd 1763 20015
httpd 20005 19891 1755

dbclean
MergeCat

如果httpd未start呢，就去csetup check吓enable左個http未？如果ready哂呢，就可以用cstop去stop哂d service然後就用cstart去start返哂．之後就係最tricky嘅地方．如果你係有用iptable嘅話，就要enable下面果d port：－

TCP 6072,6060,6051
UDP 6051, 41524

之後就應該搞掂哂，不過都係果句，本manual有差唔多等於無，你唔睇哂係唔會識用．．．無得試試吓架．．．

Arcserve for Linux

嘩！
arcserve真係好難install，事原公司班大帝話我好自把自為走去用amanda？？？
喂！我都係唔想亂駛錢啫，又係我錯？
Anyway，佢地話要用d有support嘅，有咩事都搵到人問喎．．．（我真係想問佢，點解你又用CentOS?)我真係想放長雙眼睇吓點，不過anyway，佢話要買commercial嘅咪買囉．公司嘅standard唔係Backup　Exec就係Arc　Serve，不過Backup　Exec無linux version，咁即係no choice．一路其實我都唔係好鍾意CA，不過而家無得揀，唯有硬食啦．

一收到media已經知道中招，點解睇哂三隻media都唔係for linux？問vendor佢話無理由，因為media應該得一套喎．Anyway，佢最後都ship多一隻for linux嘅trial media比我，話一樣喎．唉！要趕住install就費事浪費事間去講呢d無聊嘅問題．

到睇完個installation guide就啤一聲叫左出嚟．．．佢叫我跟住On Screen Instruction嚟做就得喎．．．無可能．Anyway，唯有跟住嚟試啦．Install完之後佢就叫我行csetup喎，唔駛講都知梗係唔work啦．

唯有去再browse吓個doc folder，睇吓有無料啦．佢個doc folder當然係分好多個sub folder去放唔同language嘅doc啦，但係搵嚟搵去都見唔到en或者eng？？不過就有一個叫做C嘅folder，估佢都係stands for "CORE"即係主要咁解．

所有嘅doc都係用number做filename，鬼識係咩number咩，好彩佢都有d sense，比左一個Table of Content嘅file．呢個當然係一個有crosslink嘅doc啦，又咁好彩比我見到一個見getting started嘅link．應該無走雞啦，睇完頭幾個chapter d content一d都唔係getting started，係似admin guide after多d，佢有無病啊？再double check下個TOC，佢竟然將個installation and configuration guide放左去Appendix後面？？？一個字：”正”！

佢入面就叫我要行一個叫/etc/profile.d/bab.sh嘅script去setup 哂所有relevant嘅environment variable先，但係行完再去行csetup一樣無嘢睇．．．我當然即刻check吓個script嘅content啦，但係又唔見有咩問題喎，都係setup同export返哂所有path，但係行完再check返d path又無set到喎．本來應該再睇吓有咩問題，不過時間嘅關係，唯有自己打返哂d command去setup d path就算，之後先再行csetup，過然work．之後都係手板眼見功夫，但係究竟用咩command行個GUI呢？又係一個字”正”．

到最後開左另一個term，當然又要行過個sh啦，不過今次我就用source去行，果然得左，都係怪我太過信佢啫。

postfix send 唔到 email ?

一路setup左個MTA幾個月，都因為種種嘅原因，包括突然有urgent task亦試過比人slide，搞到隻新嘅postfix server成幾個月都未production launch．

本來今個月諗住忙d都launch左佢唔好再拖，好彩一時心血來潮試一試，一試之後就發現中左招，唔知點解唔work．

個case係佢收到email但係sent唔出，睇個/var/log/maillog就見佢係咁鬧” postfix/qmgr[4096]: warning: connect to transport smtp: Connection refused”，上google search都見到好多post，但係無一個可以solve到我個問題，我當然已經唔理三七二十一全部都跟住嚟做．

比佢玩左一日突然間腦部震盪，叮一聲記起我自己仲有另外一隻行梗無問題嘅MTA，於是就梗係馬上login睇下兩邊個config有d咩唔同啦．一睇之下就馬上發現原來我上次比人slide嘅時候亂咁改過d config去試吓係咩問題，當然試左一日都係無發現啦，後來咪知係比人slide左一獲勁架囉，唉！但係都係自己衰，唔記得backup起d config先玩，搞到衰左都唔知，唔知衰d咩果吓慘吖嗎．

Anyway，最後發現係/etc/postfix/master.cf入面有兩行smtp都比我remark左（無錯，係兩行）．unremark左之後梗係work返啦．得返之後又比我見到/var/log/maillog入面有d怪message"postfix/smtpd[8152]: connect from unknown[123.123.123.123]"，我份人就係咁，見到d唔想見或者係唔知係咩嘅嘢就好想整左佢，呢d唔知係唔係叫排除異己呢？

但係google左一輪都無發現，最後唯有放棄．之後就再browse　http://jimsun.linxnet.com/misc/postfix-anti-UCE.txt去再harden個MTA多d，其實已經做左一定嘅UCE measure，但係還掂有時間，所以想再做多d，睇完哂之後就做左下面嘅measure．

smtpd_client_restrictions加左reject_unknown_reverse_client_hostname同埋reject_unknown_client_hostname

加呢兩個setting係想防止accept email from Unknown嘅host，包括無reverse PTR Record嘅server．但係點知加完之後，我用另外一部work緊嘅MTA send email去呢個新嘅MTA就點都唔得，/var/log/maillog就見”postfix/smtpd[8152]: NOQUEUE: reject: RCPT from unknown[123.123.123.123]: 450 4.7.1 Client host rejected: cannot find your hostname, [123.123.123.123]; from=<me@OKMTA.com> to=<me@newMTA.com> proto=SMTP helo=”．

諗來諗去都唔知點解，明明我部MTA果d DNS Record應該好齊吓架，有A, PTR同TXT(for SPF)嘅record，再者，我都無enable到SPF Checking．Google左好一陣，終於都比我搵到有人http://archives.neohapsis.com/archives/postfix/1999-q4/0199.html有d suggestion，佢話resolver唔一定搵bind，咁咪即係有機會要check吓/etc/nsswitch.conf．一睇之吓就發現"hosts: files dns"．哈哈，Resolvr果然係睇host，咁我咪試下改成"hosts: dns files”，改完咩都唔洗restart就得左．

雖然都知會對其他program有影響，好彩呢部係真係一部pure MTA所以應該無問題．而改完呢個setting仲搞掂埋"postfix/smtpd[8152]: connect from unknown[123.123.123.123]"添，其實果個post係唔關事架，但係睇吓睇吓又比我搵到d關聯．所以有時有問題不妨退一步，或者行開一吓，因為可能會比見到嘅所謂fact影響左，比佢帶完玩遊花園都未知架．

selinux好鬼煩

其實我都已經set過九次都唔係好記得個死人selinux點set．每一次setup一部新機都比佢玩一輪．為左唔好再比佢玩，我決定將今次嘅setup經歷blog低佢，以後就有得refer．

今次我嘅target係setup果套mailgraph on centOS 5，我用嘅mailgraph version係1.13，開始我仲發夢想用yum install去setup．點不知當然無啦，唯有自已去http://mailgraph.schweikert.ch download．

首先我要install好所有dependency先．

perl -MCPAN -e shell
install File::Tail
quit
wget http://oss.oetiker.ch/rrdtool/pub/rrdtool-1.2.23.tar.gz
tar zxf rrdtool-1.2.23.tar.gz
cd rrdtool-1.2.23
BUILD_DIR=/tmp/rrdbuild
INSTALL_DIR=/usr/local/rrdtool-1.2.23
./configure --prefix=$INSTALL_DIR && make && make install

之後就可以開始setup套mailgraph喇：－

wget http://mailgraph.schweikert.ch/pub/mailgraph-1.13.tar.gz
tar zxf mailgraph-1.13.tar.gz
cd mailgraph-1.13
cp mailgraph.pl /usr/local/bin

然後vi mailgraph-init
將MAIL_LOG改成"MAIL_LOG=/var/log/maillog"
cp mailgraph-init /etc/init.d

最後就vi mailgraph.cgi
將my $rrd =改成"my $rrd = '/var/lib/mailgraph.rrd'"
將my $rrd_virus =改成"my $rrd_virus = '/var/lib/mailgraph_virus.rrd'"
cp mailgraph.cgi /var/www/cgi-bin

理輪上就已經完成，可惜我enable左selinux，所就仲有嘢搞．

唔理佢嘅話你就會見到/var/log/httpd/error_log有下面嘅error message:
[Wed Aug 22 15:21:46 2007] [error] [client 99.99.99.99] ERROR: opening '/var/lib/mailgraph.rrd': Permission denied, referer: http://99.99.99.99/cgi-bin/mailgraph.cgi

而且你都會見到/var/log/messages有下面嘅error message:
Aug 22 14:57:00 server setroubleshoot: SELinux is preventing the mailgraph.cgi from using potentially mislabeled files mailgraph.rrd (var_lib_t). For complete SELinux messages. run sealert -l c961bc8c-9da8-468b-8727-8ba3eafc3517

唔做嘢當然佢唔會work啦．

[root@server ~]# ls -Z /var/lib/*.rrd
-rwxr--r-- root root user_u:object_r:var_lib_t mailgraph.rrd
-rwxr--r-- root root user_u:object_r:var_lib_t mailgraph_virus.rrd
[root@server ~]# ls -Z /var/www/cgi-bin/mailgraph.cgi
-rwxr-xr-x root root user_u:object_r:httpd_sys_script_exec_t /var/www/cgi-bin/mailgraph.cgi

一睇就知道邊度出錯啦，就係因為rrd同個cgi唔同security label囉，rrd係var_lib_t，而個cgi就係httpd_sys_script_exec_t．於是我就將/var/lib/*.rrd改個label改到一樣囉，點解唔改mailgraph.cgi？因為mailgrahp.cgi個label係control個script做得d咩，所以一改佢咪violate左個原意囉．

chcon -t httpd_user_content_t /var/lib/*.rrd
[root@server mailgraph-1.13]# ls -laZ /var/lib/*.rrd
-rwxr--r-- root root user_u:object_r:httpd_sys_content_t /var/lib/mailgraph.rrd
-rwxr--r-- root root user_u:object_r:httpd_sys_content_t /var/lib/mailgraph_virus.rrd

finally我睇到個mailgraph喇，不過都仲係有d問題，點解個graph好似停左無update到嘅?