Skip to content

刘航 / watchmen

Go to a project
Commit 1a1f167b by see.liuh@gmail.com
Options

【ADD】vproc工具增加类似top命令的查看功能 

【ADD】增加zabbix批量添加主机脚本
1 parent c64b4006
Inline Side-by-side
Showing 10 changed files with 567 additions and 3 deletions
awesome_script/README.md
...@@ -4,8 +4,33 @@ ...@@ -4,8 +4,33 @@
## vproc ## vproc
脚本说明见文档中vproc章节。 进入vproc目录，有单独的说明文档
在设备上运行时将vproc.py改名vproc， 应用程序中会执行vproc脚本 在设备上运行时将vproc.py改名vproc， 应用程序中会执行vproc脚本
vproc_testcase.py是简单的单元测试 vproc_testcase.py是简单的单元测试
## zabbix_addhost.py
批量添加zabbix主机的脚本。如果有没暴露的参数，请自己修改脚本。页面上能操作的，api都能操作。 zabbix api参考：https://www.zabbix.com/documentation/3.4/manual/api 。有这个脚本为基础，参考文档增加其他功能并不难。
脚本有help命令:
```
useage: zabbix_addhost.py [options]
[+]--baseurl=[any] | zabbix web url | default: http://192.168.9.133:9210
[+]--ipstart=[any] | ip range start | default: 192.168.66.2
[+]--ipend=[any] | ip range end | default: 192.168.66.41
[+]--user=[any] | zabbix web login user | default: Admin
[+]--passwd=[any] | zabbix web login password | default: zabbix
[+]--hostgroup=[any] | host group name | default: tx1
[+]--proxyname=[any] | porxy name | default: ""
[+]--template1=[any] | template name 1 | default: "starnet_tx1_slave_tp"
[+]--template2=[any] | template name 2 | default: "Template OS Linux"
```
- ipstart和ipend，会用ip最后一位数字生成集合。 比如ipstart=192.168.66.2，ipend=192.168.66.41，会添加66.2 66.3 66.4...66.41共40个主机
- proxyname，template1、2默认是有值的，如果想为空，需指定，比如 --proxyname="" --template2=""
awesome_script/vproc/README.md 0 → 100644
# vproc
## 概述
类似linux proc系统，以文件为媒介。应用程序可以定义多个状态值（key/value），在程序运行过程中持续更新到文件中。外部可随时读取文件查询状态值。程序开发者或运维者可以更好的掌控程序的运行状态。
所有key值均采用 app.[group1.gourp2...groupn].item方式组织。末端item为文件，记录监测项的值，前面是多级目录。比如key值vaserver.va0.cameraname，记录状态值的文件为VPORC_ROOT/vaserver/va0/cameraname。 一个监测项对应一个文件
vproc是实现上述功能的命令行工具，python编写
## vproc提供的功能
1. 应用程序调用vproc脚本写状态值。
2. 查询指定key的value值，输出到屏幕。利用此功能，实现了zabbix查询监测项的值。汇总起来统一在zabbix web上展示
3. 提供类似top命令的状态查看功能，不联动zabbix，也能实时掌控自己应用程序的运行状态。
## 使用方法
vproc工具有4个命令。
- 写：
```
# 此命令将创建文件 ROOT_VPROC/app/group1/group2/item，文件内容为value1
./vproc write --key=app.group1.gourp2.item --value=value1
```
- 读：
```
# 将ROOT_VPROC/app/group1/group2/item文件内容打印出来
./vproc read --key=app.group1.gourp2.item
```
- 重置：
```
# 删除ROOT_VPROC/app/group1/目录。相当于重置状态值
./vproc clean --path=app.group1
```
- 查看：
```
# 类似top，每隔1秒刷新输出一次状态值。参数path可以是任意级别，输出的监测项是path下的所有监测项。
./vproc watch --path=app.group1
```
## 注意事项
- 应用程序中单个监测项更新状态频率不宜过快。建议间隔时间1分钟以上
**注意**： 建议在程序（以及能代表一组的流程）启动时、关闭时重置状态，**避免旧的状态值干扰**。
## vproc 典型使用流程，拿vaserver举例:
定义监测项：
- vaserver.license
- vaserver.event_count（发出了多少事件）
- vaserver.va0.taskname（分析通道0的任务名称）
- vaserver.va0.playok （分析通道0视频是否点播成功）
### 应用程序写值
vaserver启动和退出时应调用
```
# 清理旧状态
./vproc clean --path=vaserver
```
vaserver启动、关闭一个va时（假设是编号0）应调用
```
# 清理旧状态
./vproc clean --path=vaserver.va0
```
license校验状态时
```
# 成功写1 失败写0
./vproc write --key=vaserver.license --value=1
```
执行一个任务，打开了一个通道
```
# 然后将任务信息状态更新
./vproc write --key=vaserver.va0.taskname --value=taskname1
```
视频第一帧图像回调到va0
```
./vproc write --key=vaserver.va0.playok --value=1
```
xx秒没有图像回调到va0
```
./vproc write --key=vaserver.va0.playok --value=0
```
### 调试阶段在命令行查看状态
开启一个命令行，运行
```
./vproc watch --key=vaserver
```
类似top命令，每隔1秒更新一次各状态值
### 衔接zabbix
zabbix_agentd会调用
```
./vproc read --key=vaserver.license
# 其他值
...
```
所有状态值汇总到zabbix server中，在web上统一显示所有被检测主机的状态值。vproc是集群监测的一个基石
awesome_script/vproc/vproc.py
...@@ -11,9 +11,12 @@ import sys ...@@ -11,9 +11,12 @@ import sys
import getopt import getopt
import os import os
import shutil import shutil
import time
#######################config##################### #######################config#####################
ROOT_PATH="/root/UserApp/vproc" ROOT_PATH="/root/UserApp/vproc"
# ROOT_PATH="./vproc"
version="v1.1"
################################################## ##################################################
def usage(): def usage():
...@@ -24,6 +27,7 @@ def usage(): ...@@ -24,6 +27,7 @@ def usage():
read read a key's value read read a key's value
write write a key's value write write a key's value
clean remove group of items clean remove group of items
watch display items in the command line
write command options: write command options:
[+]-k any, --key=any | status item key. [+]-k any, --key=any | status item key.
...@@ -44,6 +48,16 @@ def usage(): ...@@ -44,6 +48,16 @@ def usage():
example: example:
vproc clean --path=vaserver.va0 vproc clean --path=vaserver.va0
if executed write command example, the dir:VPROC_ROOT/vaserver/va0 would be remove. if executed write command example, the dir:VPROC_ROOT/vaserver/va0 would be remove.
watch command options:
[+]-p any, --path=any | monitoring item groups path that need to be displayed
example:
vproc watch --path=vaserver.va0
if executed write command example, the display string:
vaserver:
---- va0:
---- cameraname: camera1
---- playok: 1
""") """)
...@@ -106,11 +120,47 @@ def remove_items(key_path): ...@@ -106,11 +120,47 @@ def remove_items(key_path):
# print("rmdir {}".format(fullpath)) # print("rmdir {}".format(fullpath))
else: else:
print("no suck file or dir: {}".format(fullpath)) print("no suck file or dir: {}".format(fullpath))
def print_item(path):
key = ""
for node in path.split("/")[2:]:
if(node != "."):
key = key + node + "."
key = key[0: len(key)-1]
s=""
with open(path, 'r') as f:
strs=f.readlines()
if(len(strs)>0):
s= strs[0]
print("{}:\t\t\t\t\t\t{}".format(key, s))
def watch_items(key_paths):
for key_path in key_paths:
watch_root = ROOT_PATH
midpath = ""
paths = key_path.split('.')
paths_len = len(paths)
if(paths_len>0):
for path in paths:
watch_root = watch_root+ "/" + path
midpath = midpath+ path + "/"
if(os.path.isfile(watch_root)):
print_item(watch_root)
else:
filepaths = []
if not os.path.exists(watch_root):
print("no suck file or dir: {}".format(watch_root))
for parent_dir, dirnames, filenames in os.walk(watch_root):
for file in filenames:
file_full_name = os.path.join(parent_dir, file)
filepaths.append(file_full_name)
for path in filepaths:
print_item(path)
def main(): def main():
argc = len(sys.argv) argc = len(sys.argv)
if argc < 3 or (sys.argv[1]!= "read" and sys.argv[1]!= "write" and sys.argv[1]!= "clean"): if argc < 3 or (sys.argv[1]!= "read" and sys.argv[1]!= "write" and sys.argv[1]!= "clean" and sys.argv[1]!= "watch"):
usage() usage()
sys.exit(-1) sys.exit(-1)
...@@ -169,6 +219,23 @@ def main(): ...@@ -169,6 +219,23 @@ def main():
for index in range(len(key_path)): for index in range(len(key_path)):
remove_items(key_path[index]) remove_items(key_path[index])
sys.exit(0) sys.exit(0)
if(sys.argv[1] == "watch"):
for name,value in options:
if name in ("-h","--help"):
usage()
if name in ("-p","--path"):
key_path.append(value)
while True:
try:
os.system("clear")
os.system("date")
watch_items(key_path)
time.sleep(1)
except KeyboardInterrupt, e:
print ''
print "bye"
sys.exit(0)
sys.exit(0)
if __name__ == '__main__': if __name__ == '__main__':
......
awesome_script/vproc/vproc/vaserver/license 0 → 100644
1
\ No newline at end of file \ No newline at end of file
awesome_script/vproc/vproc/vaserver/va0/algo/status 0 → 100644
1
\ No newline at end of file \ No newline at end of file
awesome_script/vproc/vproc/vaserver/va0/cameraname 0 → 100644
camera1
\ No newline at end of file \ No newline at end of file
awesome_script/vproc/vproc/vaserver/va0/playok 0 → 100644
1
\ No newline at end of file \ No newline at end of file
awesome_script/vproc/vproc/vaserver/va1/cameraname 0 → 100644
camera2
\ No newline at end of file \ No newline at end of file
awesome_script/vproc/vproc/vaserver/va1/playok 0 → 100644
000000
\ No newline at end of file \ No newline at end of file
awesome_script/zabbix_addhost.py 0 → 100644
#!/usr/bin/python
# encoding:utf-8
#######################################################
# author: liuh 2018-10-28
# version: v1.0
# python version: 2
#######################################################
import json
import urllib2
from urllib2 import URLError
import sys
import xlrd
import getopt
class ZabbixTools:
def __init__(self, url, user, passwd):
self.url = url
self.header = {"Content-Type":"application/json"}
self.user = user
self.passwd = passwd
# 登陆验证
def user_login(self):
data = json.dumps({
"jsonrpc": "2.0",
"method": "user.login",
"params": {
"user": self.user,
"password": self.passwd
},
"id": 0
})
request = urllib2.Request(self.url, data)
for key in self.header:
request.add_header(key, self.header[key])
try:
result = urllib2.urlopen(request)
except URLError as e:
print "Auth Failed, please Check your name and password:", e.code
else:
response = json.loads(result.read())
result.close()
self.authID = response['result']
return self.authID
# 获取主机
def host_get(self,hostName):
data = json.dumps({
"jsonrpc":"2.0",
"method":"host.get",
"params":{
"output":["hostid","name"],
"filter":{"host":hostName}
},
"auth":self.user_login(),
"id":1,
})
request = urllib2.Request(self.url, data)
for key in self.header:
request.add_header(key, self.header[key])
try:
result = urllib2.urlopen(request)
except URLError as e:
if hasattr(e, 'reason'):
print 'We failed to reach a server.'
print 'Reason: ', e.reason
elif hasattr(e, 'code'):
print 'The server could not fulfill the request.'
print 'Error code: ', e.code
else:
response = json.loads(result.read())
result.close()
print "Number Of %s: " % hostName, len(response['result'])
lens=len(response['result'])
if lens > 0:
return response['result'][0]['name']
else:
return ""
# 获取主机组
def hostgroup_get(self, hostgroupName):
data = json.dumps({
"jsonrpc":"2.0",
"method":"hostgroup.get",
"params":{
"output": "extend",
"filter": {
"name": [
hostgroupName,
]
}
},
"auth":self.user_login(),
"id":1,
})
request = urllib2.Request(self.url, data)
for key in self.header:
request.add_header(key, self.header[key])
try:
result = urllib2.urlopen(request)
except URLError as e:
print "Error as ", e
else:
response = json.loads(result.read())
result.close()
lens=len(response['result'])
if lens > 0:
self.hostgroupID = response['result'][0]['groupid']
return response['result'][0]['groupid']
else:
print "no GroupGet result"
return ""
# 获取模板
def template_get(self, templateName):
data = json.dumps({
"jsonrpc":"2.0",
"method": "template.get",
"params": {
"output": "extend",
"filter": {
"host": [
templateName,
]
}
},
"auth":self.user_login(),
"id":1,
})
request = urllib2.Request(self.url, data)
for key in self.header:
request.add_header(key, self.header[key])
try:
result = urllib2.urlopen(request)
except URLError as e:
print "Error as ", e
else:
response = json.loads(result.read())
result.close()
self.templateID = response['result'][0]['templateid']
return response['result'][0]['templateid']
# 创建host
#def host_create(self, hostName,visibleName,hostIp,dnsName,proxyName, hostgroupName, templateName1, templateName2):
def host_create(self, hostName,visibleName,hostIp, hostgroupName, proxyName, templateName1, templateName2):
root_dict = {
"jsonrpc":"2.0",
"method":"host.create",
"params":{
"host": hostName,
"name": visibleName,
"interfaces": [
{
"type": 1,
"main": 1,
"useip": 1,
"ip": hostIp,
"dns": "",
"port": "10050"
}
],
"groups": [
{
"groupid": self.hostgroup_get(hostgroupName)
}
],
# "templates": [
# {
# "templateid": self.template_get(templateName1)
# },
# {
# "templateid": self.template_get(templateName2)
# }
# ],
},
"auth": self.user_login(),
"id":1
}
if(proxyName != ""):
root_dict["params"]["proxy_hostid"] = self.proxy_get(proxyName)
templates = []
if(templateName1 != ""):
templates.append({"templateid": self.template_get(templateName1)})
if(templateName2 != ""):
templates.append({"templateid": self.template_get(templateName2)})
if len(templates) > 0:
root_dict["params"]["templates"] = templates
data = json.dumps(root_dict)
request = urllib2.Request(self.url, data)
for key in self.header:
request.add_header(key, self.header[key])
try:
result = urllib2.urlopen(request)
except URLError as e:
print "Error as ", e
else:
response = json.loads(result.read())
result.close()
print "host : %s is created! id is %s\n" % (hostip, response['result']['hostids'][0])
self.hostid = response['result']['hostids']
return response['result']['hostids']
# zabbix_proxy获取函数
def proxy_get(self, ProxyName):
data = json.dumps({
"jsonrpc":"2.0",
"method": "proxy.get",
"params": {
"output": "extend",
"selectInterface": "extend",
"filter": {
"host": [ ProxyName, ]
}
},
"auth":self.user_login(),
"id":1,
})
request = urllib2.Request(self.url, data)
for key in self.header:
request.add_header(key, self.header[key])
try:
result = urllib2.urlopen(request)
except URLError as e:
print "Error as ", e
else:
response = json.loads(result.read())
result.close()
self.templateID = response['result'][0]['proxyid']
return response['result'][0]['proxyid']
def usage():
print("""
useage: zabbix_addhost.py [options]
[+]--baseurl=[any] | zabbix web url | default: http://192.168.9.133:9210
[+]--ipstart=[any] | ip range start | default: 192.168.66.2
[+]--ipend=[any] | ip range end | default: 192.168.66.41
[+]--user=[any] | zabbix web login user | default: Admin
[+]--passwd=[any] | zabbix web login password | default: zabbix
[+]--hostgroup=[any] | host group name | default: tx1
[+]--proxyname=[any] | porxy name | default: ""
[+]--template1=[any] | template name 1 | default: "starnet_tx1_slave_tp"
[+]--template2=[any] | template name 2 | default: "Template OS Linux"
""")
# 程序的入口
if __name__ == "__main__":
try:
options,args = getopt.getopt(sys.argv[1:],"",["help","baseurl=","ipstart=", "ipend=", "user=",\
"passwd=", "hostgroup=", "proxyname=", "template1=", "template2="])
except getopt.GetoptError:
print("error: invalid args")
usage()
sys.exit(-1)
baseurl = "http://192.168.9.133:9210"
ipstart = "192.168.66.2"
ipend = "192.168.66.2"
user = "Admin"
passwd = "zabbix"
hostgroup = "tx1"
proxyname = ""
template1 = "starnet_tx1_slave_tp"
template2 = "Template OS Linux"
for name,value in options:
if name in ("--help"):
usage()
if name in ("--baseurl"):
baseurl = value
if name in ("--ipstart"):
ipstart = value
if name in ("--ipend"):
ipend = value
if name in ("--user"):
user = value
if name in ("--passwd"):
passwd = value
if name in ("--hostgroup"):
hostgroup = value
if name in ("--proxyname"):
proxyname = value
if name in ("--template1"):
template1 = value
if name in ("--template2"):
template2 = value
print("baseurl={}\n ipstart={}\n ipend={}\n user={}\n passwd={}\n hostgroup={}\n proxyname={}\n template1={}\n template2={}\n".format(\
baseurl, ipstart, ipend, user, passwd, hostgroup, proxyname, template1, template2))
url=baseurl+"/api_jsonrpc.php"
# # 实例化ZabbixTools对象
test = ZabbixTools(url, user, passwd)
sstrs = ipstart.split(".")
estrs = ipend.split(".")
ipbase = "{}.{}.{}".format(sstrs[0], sstrs[1], sstrs[2])
intstart = int(sstrs[3])
intend = int(estrs[3])
# ipbase = "192.168.66"
for i in range(intstart,intend+1):
hostip = "{}.{}".format(ipbase,i)
print(hostip)
test.host_create( hostip,hostip,hostip, hostgroup, proxyname, template1, template2)
Markdown is supported
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!