我目前正在使用一个使用 mongodb 返回器的 saltstack 实现，我正在尝试查询存储在其中的结果以用于报告目的。不幸的是，返回者将信息写入数据库的方式如下所示:

db.hostname21.find({"20140421035007474763" : {$exists : 1}}, {_id: 0}).pretty()

{
     "fun" : "state.sls",
     "20140421035007474763" : {
            "file_|-Get-Logins-Stats_|-/scripts/server_logins_stats_|-managed" : {
                    "comment" : "File /scripts/server_logins_stats is in the correct state",
                    "__run_num__" : 2,
                    "changes" : {

                    },
                    "name" : "/scripts/server_logins_stats",
                    "result" : true
            },
            "service_|-Logger_|-rsyslog_|-running" : {
                    "comment" : "Service rsyslog is already enabled, and is in the desired state",
                    "__run_num__" : 1,
                    "changes" : {

                    },
                    "name" : "rsyslog",
                    "result" : true
            },
            "cmd_|-Run_Script_|-/scripts/server_logins_stats_|-run" : {
                    "comment" : "Command \"/scripts/server_logins_stats\" run",
                    "__run_num__" : 4,
                    "changes" : {
                            "pid" : 20899,
                            "retcode" : 0,
                            "stderr" : "",
                            "stdout" : "0"
                    },
                    "name" : "/scripts/server_logins_stats",
                    "result" : true
            },
            "cron_|-Schedule_Run_|-/scripts/server_logins_stats_|-present" : {
                    "comment" : "Cron /scripts/server_logins_stats already present",
                    "__run_num__" : 3,
                    "changes" : {

                    },
                    "name" : "/scripts/server_logins_stats",
                    "result" : true
            },
            "pkg_|-nc_|-nc_|-installed" : {
                    "comment" : "Package nc is already installed",
                    "__run_num__" : 0,
                    "changes" : {

                    },
                    "name" : "nc",
                    "result" : true
            }
    }
}

正如您在此处看到的那样，键不断变化，而不是使用一个键来标识每个脚本。然而，我发现失败的格式非常一致，除了它们没有任何键来标识它只是一个字符串数组的失败:

       "20140421041507478163" : [
            "Pillar failed to render with the following messages:",
            "Specified SLS 'globals' in environment 'Production' is not available on the salt master"
       ],
       "fun" : "state.sls"

所以最终我想要做的是能够通过识别作业、主机和失败的性质以及随着时间的推移成功和失败的总数来报告每次失败。如果您注意到每个主机(minion)都创建了自己的集合。所以我有一个 python 脚本，它将遍历集合以确定作业是否实际在该主机上执行:

import datetime
import pymongo


#hosts = ["mongoDBStaging", "mongoDBUAT"]
hosts = ["mongodbuat"]

for host in hosts:
    conn = pymongo.Connection(host)
    mdb = conn['salt']
    collections = set(mdb.collection_names())


    hosts = []
    jids = []

    # for every collection, identify whether it is a host or a job
    for c in collections:   
        # if the collection is a host add it to the host array
        if not (len(c) == 20 and int(c)):
            #print "{0} is a host".format(c)
            hosts.append(c)
        # other  wise add it to the job array
        else:
            #print "{0} is a jid".format(c)
            jids.append(c)


    for h in hosts:
        # for every job in a host connect to that collection 
        # and search for the job id to see if it exists 
        # and what its return was so we can report on that info
        for j in jids:
            coll = conn['salt'][h]
            #print "%s collection, %s jid" % (coll, j)
            for doc in coll.find({j: {'$exists': True}, "fun": "state.sls"}, {"_id": 0}):
                print "{0}".format(coll)
                print "{0} is a doc".format(doc)

但我正在努力查询结果以实际确定它是否成功。通过能够提取返回的文档元素以读入每个文档的结果。

如果有人对我如何始终如一地查询键不断变化以获得真正有用​​的结果有任何建议。

最佳答案

如果有人想知道。我使用以下 python 代码解决了自己的问题。绝对不是性能或充分利用 mongodb 的最佳选择，但它确实有效。考虑到 shell 中的查询非常有限，我可能会建议更新 salt 返回器以更好地使用 mongodb。

import datetime
import pymongo
import json
import re

hosts = ["mongodbuat"]

# initialize failures and successes
failures = 0
successes = 0

for host in hosts:
    conn = pymongo.Connection(host)
    mdb = conn['salt']
    collections = set(mdb.collection_names())

    hosts = []
    jids = []

    # for every collection, identify whether it is a host or a job
    for c in collections:   
        # if the collection is a host add it to the host array
        if not (len(c) == 20 and int(c)):
            hosts.append(c)
        # otherwise add it to the job array
        else:
            jids.append(c)


    for h in hosts:
        # for every job in a host connect to that collection 
        # and search for the job id to see if it exists 
        # and what its return was so we can report on that info
        for j in jids:
            coll = conn['salt'][h]

            # search for the json documents returned from mongodb
            # if the jobid exists in that host
            for doc in coll.find({j: {'$exists': True}, "fun": "state.sls"}, {"_id": 0}):

                # if the jobid exists find the host name in a readable format
                c = str(coll)
                thishost = ''
                match = re.search('(\w+)\.spottrading\.com',c)
                if match:
                    thishost = match.group(1)

                # search the document returned in the form of a dictionary for
                # the states you want to report on
                for jobid, states in doc.iteritems():
                    if re.search('\d+', jobid):
                        print '\njob id =', jobid
                        if isinstance(states, list):
                            print states
                            failures += 1
                        elif isinstance(states, dict):
                            for job, data in states.iteritems():
                                print '\tjob: {0}, result: {1}'.format(job, data[u'result'])
                            successes += 1

print "{0} successes, {1} failures".format(successes, failures)                         

关于python - mongodb查询其中键可变的数据并使用python聚合结果，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/23271979/