My script runs in the background after a period of time not to grab the data

1

Hello, everyone!

Use python scripts fetching data after a period of time, the connection is interrupted, the browser cannot access.I can’t find out why?And fetching data section is chaotic, lead to cannot be stored into the database.

Help me!Thank you!

System: Centos6.4
Version: 0.13

my script：

#coding=utf-8
import os
import pymongo
import time
from libmproxy import flow, proxy
from libmproxy.proxy.server import ProxyServer
from libmproxy.flow import FlowWriter


class MyMaster(flow.FlowMaster):
    def run(self):
        self.db_init()
        try:
            flow.FlowMaster.run(self)
        except KeyboardInterrupt:
            self.shutdown()

    def handle_request(self, f):
        f = flow.FlowMaster.handle_request(self, f)
        if f:
            print f.request.url
            f.reply()
        return f

    def handle_response(self, f):
        f = flow.FlowMaster.handle_response(self, f)
        if f:
            self.db_insert(f)
            f.reply()
        return f

    def db_init(self):
        self.client = pymongo.MongoClient('127.0.0.1', 27017)
        self.db = self.client['scan']
        self.coll = self.db['scan_res']
        return

    def db_insert(self, flow):
        insert_dict = {}
        if flow:
            insert_dict = {
                'time': time.time(),
                'request': {
                    'url': flow.request.url,
                    'scheme': flow.request.scheme,
                    'path': flow.request.path,
                    'method': flow.request.method
                },
                'response': {
                    'msg': flow.response.msg,
                    'code': flow.response.code,
                    'header': dict(flow.response.headers),
                    'content': flow.response.content
                }
            }
            self.coll.insert(insert_dict)
        else:
            return
    
def test_start():
    config = proxy.ProxyConfig(
        port=8080,
        # use ~/.mitmproxy/mitmproxy-ca.pem as default CA file.
        cadir="~/.mitmproxy/",
    )
    state = flow.State()
    server = ProxyServer(config)
    m = MyMaster(server, state)
    m.run()

test_start()
2

Hi,

You are using a very outdated version of mitmproxy - if you want any help, I’d recommend upgrading first.

3

Thank you very much for your answer！

Listen to your advice I’m going to learn Python3.5 and new mitmproxy version.
But I want to achieve similar functionality in the new version also need your guidance.
Excuse me is there a reference material,or in the documentation.

Thank you again for!