注意:测试方法或者推算思路,可能有问题,欢迎批评与指正。
Mongodb的版本:
执行命令mongod --version得到 "version" :
"3.0.1",
|
myNote
|
Master
|
slave
|
Test-server
|
|
硬件配置:
|
|
|
2核心
2个逻辑cpu
主频2.2GHz
|
6核心
24个逻辑cpu
主频2.0GHz
|
|
4核心
8个逻辑cpu
2.1GHz
|
|
测试单个请求的处理时间
|
|
数据量为3W个相同点
|
109ms
|
199ms
|
|
212ms
|
|
数据量为2W个相同点
|
76ms
|
131ms
|
|
129ms
|
|
数据量为1W个相同点
|
36ms
|
66ms
|
|
63ms
|
|
数据量为1W个相同点, 测试并发多请求的处理时间 。(极端人口密集的情况)
|
并发10
|
149ms
|
88ms
|
|
122ms
|
|
并发20
|
308ms
|
135ms
|
|
213ms
|
|
并发 30
|
454ms
|
189ms
|
|
333ms
|
|
并发
50
|
778ms
|
246ms
|
|
503ms
|
|
并发
100
|
1634ms
|
585ms
|
|
724ms
|
|
并发
200
|
|
1035ms
|
|
1375ms
|
|
数据量为100W个均匀分布的点,测试并发多请求的处理时间。(均匀分布的情况)
|
|
并发
1000
|
|
10ms以下。测试机cpu耗尽。服务端cpu峰值20%
|
|
|
|
推算:能处理并发为1000*5*24约12W/S;
参考资料:1亿条记录的MongoDB数据库随机查询性能测试 http://www.jb51.net/article/44693.htm
|
|
数据量为1W个点(180,180) ,测试搜索(0,0)附近的点的处理时间.(极端人口稀疏的情况)
|
|
1W
|
|
59ms
|
|
|
|
10W
|
|
603ms
|
|
|
|
20W
|
|
962ms
|
|
|
|
30W
|
|
1241ms
|
|
|
|
结论:
写性能:写数据的速度不稳定,表现为写入一万条数据
可能耗时280至320毫秒;写的极限是,2M/s,频率约2W/s;
读性能:读数据的速度表现稳定,并发一千下,每个请求的处理耗时在10ms以下;24个cpu的峰值为20%,推论出频率约10W/s;
#!/usr/bin/python
import pymongo
from pymongo import MongoClient, GEO2D
import time
import random
from bson.son import SON
import threading
from multiprocessing import Process
x_min_num=30
x_max_num=31
y_min_num=179
y_max_num=180
def mongodb_connect():
connection = pymongo.Connection("192.168.0.102",27017)
#db = connection.admin
#db.authenticate("superuser","pwd") #用户认证
db = connection.test
return db
def mongodb_save(collections,document):
collections.insert(document)
'''
db.users.ensureIndex({"gps":"2d"},{"min":-180,"max":180})
'''
def mongodb_createCollection():
print "mongodb_createCollection"
db = mongodb_connect()
my_collection = db.users
my_collection.create_index([("gps", GEO2D)])
#my_collection.create_index("mike")
# my_collection.create_index([("mike", pymongo.DESCENDING), ("eliot", pymongo.ASCENDING)])
#my_collection.ensure_index([("gps",pymongo.GEO2D),("min",-180),("max",180)],background=True)
def mongodb_destroyCollection():
db = mongodb_connect()
my_collection = db.users
my_collection.drop()
print "mongodb_destroyCollection"
#my_collection.drop_index([("gps", GEO2D)])
def mongodb_insert_1w_data():
db = mongodb_connect()
my_collection = db.users
doc_list=list()
for i in xrange(10000):
doc=dict()
x=random.uniform(x_min_num,x_max_num)
y=random.uniform(y_min_num,y_max_num)
doc["account"]=str(100+i)
doc["gps"]=[ x,y ]
doc_list.append(doc)
begin_time = time.time()
my_collection.insert(doc_list)
end_time = time.time()
print "my_collection.insert_many done:",end_time-begin_time
def mongodb_findNear1km( location ):
db = mongodb_connect()
my_collection = db.users
begin_time = time.time()
doc_list = my_collection.find( {"gps": {"$near": location}} ).limit(100)
end_time = time.time()
# for doc in doc_list:
# print "gps:",doc["gps"]
# print "----------------"
#
# print "mongodb_findNear1km begin_time:",begin_time
# print "mongodb_findNear1km end_time:",end_time
#print "mongodb_findNear1km done:",end_time-begin_time ,begin_time,end_time
class WorkerThread(threading.Thread):
def __init__(self,worker_index):
threading.Thread.__init__(self)
self.worker_index=worker_index
def run(self):
print "*"*20
x=random.uniform(x_min_num,x_max_num)
y=random.uniform(y_min_num,y_max_num)
location= [x, y]
mongodb_findNear1km(location)
print "worker:",self.worker_index,location
def mongodb_insert_some_data(how_many):
begin_time = time.time()
for i in xrange(how_many):
mongodb_insert_1w_data()
end_time = time.time()
print "mongodb_insert_some_data done:",end_time-begin_time
def mongodb_test_concurrentQueryRequest(concurrent_num):
print "mongodb_test_concurrentQueryRequest:", concurrent_num
for i in xrange(concurrent_num):
worker = WorkerThread(i)
worker.setDaemon(True)
worker.start()
def sleeper(name, seconds):
mongodb_test_concurrentQueryRequest(100)
time.sleep(1)
def testMore(num):
process_list=list()
for i in xrange(num):
p = Process(target=sleeper, args=('bob', 5))
p.start()
process_list.append(p)
#p.join()
for p in process_list:
p.join()
def mongodb_putdata():
mongodb_destroyCollection()
mongodb_createCollection()
mongodb_insert_some_data(10)
def testone():
x=random.uniform(x_min_num,x_max_num)
y=random.uniform(y_min_num,y_max_num)
location= [x, y]
mongodb_findNear1km(location)
def main():
print "main"
mongodb_putdata()
#testone()
return
#testMore(1)
mongodb_test_concurrentQueryRequest(100)
count=0
while 1:
time.sleep(1)
count +=1
#testone()
print count
if count > 100:
print "count 100"
break
main()