好吧,一个makefile的初学者学习skynet中

1、第一步就是项目的组织方式。skynet经常要更新,所以要尽量避免在自己项目共用同一个git。从群中和云风的文章中了解到,使用submodule引用skynet是一种比较理想的方式。我一个makefile基本为0的人要理解submodule的编写很不容易,而且也不知道过程对不对,但现在至少已经可以初步达到目标了。

# hello game
#

.PHONY : all clean skynet

CC ?= gcc
PLAT ?=linux

LUA_CLIB_PATH=/opt/git/hellogame/luaclib
CSERVICE_PATH=/opt/git/hellogame/cservice

SKYNET_BUILD_PATH ?=/opt/git/hellogame

all : skynet

skynet :
        cd module_skynet && $(MAKE) CC=$(CC) SKYNET_BUILD_PATH=$(SKYNET_BUILD_PATH) LUA_CLIB_PATH=$(LUA_CLIB_PATH) CSERVICE_PATH=$(CSERVICE_PATH) linux

clean :
        cd module_skynet && $(MAKE) CC=$(CC) SKYNET_BUILD_PATH=$(SKYNET_BUILD_PATH) LUA_CLIB_PATH=$(LUA_CLIB_PATH) CSERVICE_PATH=$(CSERVICE_PATH) clean

写的很差,先用着吧。module_skynet是引用的skynet

2、复制module_skynet/example/config到本地目录。

然后修改config中的lua地址:

root = "./"
thread = 8
logger = nil
harbor = 1
address = "127.0.0.1:2526"
master = "127.0.0.1:2013"
start = "main"  -- main script
bootstrap = "snlua bootstrap"   -- The service for bootstrap
standalone = "0.0.0.0:2013"
luaservice = root.."service/?.lua;"..root.."test/?.lua;"..root.."examples/?.lua;".."module_skynet/service/?.lua;"..root.."module_skynet/test/?.lua;"..root.."module_skynet/examples/?.lua"
lualoader = "lualib/loader.lua"
-- preload = "./examples/preload.lua"   -- run preload.lua before every lua service run
snax = root.."examples/?.lua;"..root.."test/?.lua"
cpath = root.."cservice/?.so"
-- daemon = "./skynet.pid"

只修改其中 luaservice的地址。

3、启动 ./skynet config,提示找不到lualib/loader.lua,又建立了一个软连接指向module_skynet/lualib,可以顺利解决问题。

4、以上方式不保证是最合适的,但至少可以进行开发了。

mysql 中的date 0 处理

jdbc mysql遇到日期为0时即报异常。

这里需要在jdbc.url上多传一个参数 zeroDateTimeBehavior=convertToNull

用来告诉jdbc驱动,遇到date为0是设置为null。

安装ganglia

#安装ganglia

yum install zlib-devel freetype-devel libart_lgpl-devel libpng-devel rrdtool expat apr apr-util gcc-c++ rrdtool-devel  apr-devel apr-util-devel

#安装libconfuse

wget http://download.savannah.gnu.org/releases/confuse/confuse-2.7.tar.gz
tar xvfz confuse-2.7.tar.gz
cd confuse-2.7
./configure CFLAGS=-fPIC
make
make install

#安装ganglia
./configure –with-gmetad –with-python –enable-gexec –with-libconfuse=/usr/local/confuse –with-libexpat=/usr/local/expat  –sysconfdir=/etc/ganglia
make
make install

#创建rrdtool数据目录:

mkdir -p /var/lib/ganglia/rrds
mkdir -p /var/lib/ganglia/dwoo
chown -R apache:apache /var/lib/ganglia

#修改/etc/ganglia/gmetad.conf文件:
vim /etc/ganglia/gmetad.conf
data_source “suc” localhost
setuid_username “apache”

#启动 gmetad
cp -f gmetad/gmetad.init /etc/init.d/gmetad
cp -f /usr/local/sbin/gmetad /usr/sbin/gmetad
chkconfig –add gmetad
service gmetad start

#通过telnet localhost 8651验证gmetad是否正常

#客户端配置 gmond节点
cp -f gmond/gmond.init /etc/init.d/gmond
cp -f /usr/local/sbin/gmond /usr/sbin/gmond
chkconfig –add gmond
gmond –default_config > /etc/ganglia/gmond.conf
#修改/etc/ganglia/gmond.conf配置文件:
cluster {
name=”suc”
owner=”apache”
latlong=”unspecified”
url=”unspecified”
}

service gmond start
通过telnet localhost 8649验证gmond是否正常

ganglia-web可以允许在web中浏览数据,是使用php写的,网上大多数文章写的放在apache httpd中运行,实际上nginx一样可以运行。

脚本远程上传文件

#!/bin/bash
local_dir=$1
target_dir=$1
user=root
password=password
serverip=ip

lftp -c "open -u $user,$password sftp://$serverip ; mirror -R --delete $local_dir $target_dir"

今天想测试map的读取key的性能,结果遇到无法解答的问题。

代码如下:

如果注释掉test2(),启动test2_1(),则test()消耗的时间大幅减少,不知原因。

import java.util.HashMap;
import java.util.IdentityHashMap;
import java.util.Map;

public class Test4 {
	private static Map<Integer, Integer> map1 = new HashMap<Integer, Integer>();
	private static Map<String, String> map2 = new HashMap<String, String>();
	private static Map<String, String> map3 = new IdentityHashMap<String, String>();
	private static String testKey = "helloworld";
	private static long time;
	private static long time2;
	private static Integer a = 1500;

	public static void main(String[] args) {
		map3.put(testKey, testKey);
		for (int m = 1000; m < 1010; m++) {
			map1.put(m, m);
			map2.put("" + m, "" + m);
			map3.put("" + m, "" + m);
		}
		for (int mm = 0; mm < 100; mm++) {
			test1();
			test2();
			// test2_1();
		}
	}

	public static void test1() {
		time = System.nanoTime();
		for (int m = 0; m < 1000000000; m++) {
			map1.get(a);
		}
		time2 = System.nanoTime();
		System.out.println("数字:" + (time2 - time));
	}

	public static void test2() {
		time = System.nanoTime();
		for (int m = 0; m < 1000000000; m++) {
			map2.get("hello");
		}
		time2 = System.nanoTime();
		System.out.println("字符:" + (time2 - time));
	}

	public static void test2_1() {
		time = System.nanoTime();
		for (int m = 0; m < 1000000000; m++) {
			map2.get("hello222222222222222222222222222222222222222222222222222222222222222222222222222222222222222");
		}
		time2 = System.nanoTime();
		System.out.println("字符:" + (time2 - time));
	}

	public static void test3() {
		time = System.nanoTime();
		for (int m = 0; m < 1000000000; m++) {
			map3.get(testKey);
		}
		time2 = System.nanoTime();
		System.out.println("字符3:" + (time2 - time));
	}
}

loganalyzer3.6.5的一个bug

从昨天开始搞loganalyzer,一直无法显示自己mysql中的日志。

日志表是自定义的。

+---------+---------------------+------+-----+-------------------+----------------+
| Field   | Type                | Null | Key | Default           | Extra          |
+---------+---------------------+------+-----+-------------------+----------------+
| logId   | bigint(20) unsigned | NO   | PRI | NULL              | auto_increment | 
| account | char(64)            | NO   |     |                   |                | 
| server  | int(10) unsigned    | NO   |     | 0                 |                | 
| device  | char(64)            | NO   |     |                   |                | 
| error   | blob                | YES  |     | NULL              |                | 
| uID     | int(11)             | YES  |     | 0                 |                | 
| date    | timestamp           | NO   |     | CURRENT_TIMESTAMP |                | 
+---------+---------------------+------+-----+-------------------+----------------+

自己定义datasource,field,view,dbmapping,一切都OK。但就是显示有问题。

如下图所示少了uID那一列的数据。

google了N多资料都无解。

网上也有一个相同现象的帖子,也没有找到原因:http://kb.monitorware.com/reading-custom-mysql-table-t11919.html

一气之外感觉自己debug可能能找到原因。

在classes/logstreamdb.class.php 中

修改readnext方法如下:

 if ( $content['MaxExecutionTime'] > 0 && $scriptruntime > ($content['MaxExecutionTime']-2) )
                        {
                                // This may display a warning message, so the user knows we stopped reading records because of the script timeout.
                                $content['logstream_warning'] = "false";

                                // Run optional Message Parsers now
                                if ( isset($arrProperitesOut[SYSLOG_MESSAGE]) )
                                {
                                        $retParser = $this->_logStreamConfigObj->ProcessMsgParsers($arrProperitesOut[SYSLOG_MESSAGE], $arrProperitesOut);

                                        // Check if we have to skip the message!
                                        if ( $retParser == ERROR_MSG_SKIPMESSAGE )
                                                $ret = $retParser;
                                }

                                // Set uID to the PropertiesOut! //DEBUG -> $this->_currentRecordNum;
                                echo "<br/>";
                                echo $dbmapping[$szTableType];
                                foreach($dbmapping[$szTableType] as $key => $val) {
                                   echo "|".$key."|".$val, '<br>';
                                }
                                foreach($dbmapping[$szTableType]['DBMAPPINGS'] as $key => $val) {
                                   echo "|_|".$key."|".$val, '<br>';
                                }

                                foreach($this->bufferedRecords[$this->_currentRecordNum] as $key => $val) {
                                   echo "|||".$key."|".$val,"<br/>";
                                }

                                echo "SYSLOG_UID:".SYSLOG_UID."<br/>";
                                echo "hello ". $dbmapping[$szTableType]['DBMAPPINGS']['uID']."world";
                                echo "hello ". $this->bufferedRecords[$this->_currentRecordNum]['uid']."world";
                                echo "<br/>";
                                $uID = $arrProperitesOut[SYSLOG_UID] = $this->bufferedRecords[$this->_currentRecordNum][$dbmapping[$szTableType]['DBMAPPINGS'][SYSLOG_UID]];

                                // Increment $_currentRecordNum
                                $this->_currentRecordNum++;
                        }

把debug信息输出后,才发现问题。

uID对应数据中的logId字段,但在this->bufferedRecords中的key全是小写的。你妹的,各个文档上谁也没写要注意这么一个东西。dbmapping直接修改uID改为logid小写后,日志查看bug解决了。

正常的显示:

 

 

syslog接收客户端发送的日志

仅仅是这一个简单的功能,不想再重新再用java重新做。

直接用nginx+php往syslog中写,syslog往mysql中输出,使用log查看工具查看日志。

php中打开syslog输出:

1、编辑php.ini

取消下面一行的注释

error_log = syslog

2、重启nginx+php

3、写php测试一下:

<?php
        openlog('php', LOG_CONS | LOG_NDELAY | LOG_PID, LOG_USER | LOG_PERROR);
        syslog(LOG_INFO, 'hello php');
?>

4、在/var/log/message中可以看到输出:

php[21641]:  hello php
php[21642]:  hello php
php[21643]:  hello php

rsyslog接收日志写入mysql配置

$EscapeControlCharactersOnReceive off

$template TraditionalFormat2,"%msg%\n"
$template TraditionalFormat3,"%syslogtag% %msg%\n"
$template TraditionalFormat4,"%syslogtag% %msg%\n"
$template StdSQLFormat,"insert into bi_action(id,serverId,`date`,`%msg:R,ERE,1,DFLT:\w+ \w+ (\w+)--end%`) values(%msg:R,ERE,1,DFLT:(\w+)--end%,%msg:R,ERE,1,DFLT:\w+ (\w+) \w+--end%,%$year%%$month%%$day%,1) on duplicate key update `%msg:R,ERE,1,DFLT:\w+ \w+ (\w+)--end%`=`%msg:R,ERE,1,DFLT:\w+ \w+ (\w+)--end%`+1  ",SQL



        if re_match($syslogtag,'^moon[0-9]+-[a-zA-Z_]+.log$') then {
                $ActionFileDefaultTemplate TraditionalFormat2
                $template DynFile2,"/opt/logs/router-log/%syslogtag:R,ERE,1,DFLT:(.*)-.*.log--end%/%timegenerated:1:10:date-rfc3339%/%syslogtag:R,ERE,1,DFLT:.*-(.*).log--end%.log"
                *.* -?DynFile2
                $ModLoad ommysql
                *.* :ommysql:127.0.0.1,Syslog,root,
                stop
        } else if re_match($syslogtag,'^moon[0-9]+$') then {
                        $ActionFileDefaultTemplate TraditionalFormat3
                        $template DynFile3,"/opt/logs/router-log/moon-alert.log"
                        *.* -?DynFile3
                        stop
        } else if re_match($syslogtag,'^moon[0-9]+\\-BI$') then {
                        $ActionFileDefaultTemplate TraditionalFormat4
                        $template DynFile4,"/opt/logs/router-log/moon-BI.log"
                        *.* -?DynFile4
                        *.*     >127.0.0.1,Syslog,root,;StdSQLFormat

                        stop
        }

java visualvm jstatd启动时遇到的一个错误

今天启动jstatd时一直报下面的错误:

root@cvs jstatd]# jstatd -J-Djava.security.policy=/opt/joycube/jstatd/tools.policy
Could not contact registry
Connection refused to host: 192.168.17.12; nested exception is:
        java.net.ConnectException: Connection timed out
java.rmi.ConnectException: Connection refused to host: 192.168.17.12; nested exception is:
        java.net.ConnectException: Connection timed out
        at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
        at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
        at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
        at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:341)
        at sun.rmi.registry.RegistryImpl_Stub.rebind(Unknown Source)
        at java.rmi.Naming.rebind(Naming.java:177)
        at sun.tools.jstatd.Jstatd.bind(Jstatd.java:57)
        at sun.tools.jstatd.Jstatd.bind(Jstatd.java:66)
        at sun.tools.jstatd.Jstatd.main(Jstatd.java:143)
Caused by: java.net.ConnectException: Connection timed out
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:579)
        at java.net.Socket.connect(Socket.java:528)
        at java.net.Socket.<init>(Socket.java:425)
        at java.net.Socket.<init>(Socket.java:208)
        at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40)
        at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:147)
        at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613)
        ... 8 more

上面提示的ip很奇怪。这台机器的ip应该是10.234.10.12

运行hostname -i,显示为192.168.17.12

这是错误的IP,打开 /etc/hosts,发现里面配置了本机名cvs解析为192.158.17.12,这台机器以前换过ip,修改为10.234.10.12。

ok,jstatd可以正常运行了。

mingw遇到的一个问题

自己的写一小段代码:

#include <stdio.h>
#include <string.h>
#include <lua.h>

int main(int argc,char* argv[]){
        lua_State* L=luaL_newstate();
        char tmp[1024];
        while(!feof(stdin)){
                fgets(tmp,sizeof(tmp),stdin);
                printf("%4lu %s\n",strlen(tmp),tmp);
        }
        return 0;
}

在win7 64位,mingw下编译总是报警告:

format '%lu' expects argument of type 'long unsigned int', but argument 2 has type 'size_t' [-Wformat=]    test3.c    /test7/src    line 18    C/C++ Problem

google了一下,试了各种方法,也无结果。
突然灵感一来,想起来mingw是32位系统,难道是因为这个原因,下载mingw64试下。

安装mingw64:http://sourceforge.net/projects/mingwbuilds/