Storing and Purging traffic on HDDs

help
1

So I read a few posts about sending decrypted traffic to another network IP for storage and analysis, however, given that it requires additional scripting and our company is so small that we cant afford, i was wondering of a basic alternative:

  1. Can I store traffic on the same server MITMPROXY is running? If so, is there a way to set limites etc?
  2. Is it possible to purge after X amount of GBs used or days so it will be cyclical?

We want to make sure that some of the applications that we use send what is necessary and in case of any issues while working on them we can bring it up to the other party.

Very excited about MITMproxy capabilities and a network administrator.

2

Hi,

I’m not 100% clear on the end solution you have in mind, but based on the question you should consider looking into logrotate to handle your logfiles, you can specify rotation of the log based on time, size etc. You can also compress them for shipping once they are rotated off.

Anton.

3

You can look here to store data in a MongoDB istance.
Purge is not implemented and you have to do by yourself.

4

Thank you for replay, however, given that i’m quite new to this, was wondering if you can expand more on:

Anton: Couldn’t find ‘logrotate’, is it part of mitmproxy? can you point me in its direction?

Gigarum, regarding MongoDB i appreciate the information. Never heard about, so I’ll look into it. by the way, is there some tutorial on how to make mitmproxy and mongodb work together?

5

Hi @jamthejame,

Like @Gigiarum said, you could also use mongodb and use mongodump to do your archiving etc. Depends on the solution you have in mind.

I’m not sure what OS you are on, but logrotate is a linux utility designed to handle any kind of log file you tell it about in its configuration. I.e. if you have an application that logs to /var/log/blah.log, you can configure logrotate to watch that file, and apply rotation or archiving based on the rules you set, for example:

$ cat blah.conf
/var/log/blah.log {
        daily
        copytruncate
        rotate 7
        compress
}

There are quite a few resources out there for it. If you google ‘logrotate’ you will see lots of examples.

Anton.

6

Anton,

I’m using Lubuntu, so mimtmweb doesn’t work on it, given that it’s a light system and probably missing some libraries, however, that’s not my primary concern.

Given you in-depth knowledge, can you please clarify the following, can mitmproxy effective decipher SSL pre-shared key traffic via 443 from example like a platform MetaTrader 4.

I read the whole doc on mitm but still can’t figure out if something like that can be done.

7

You could try installing it, something like

sudo apt-get install logrotate

on Ubuntu based systems.

Sorry I’m not familiar enough with mitmproxy to answer your PSK question, but I can’t imagine you will be able to achieve this without the key and some coding on your side. @mhils would be better positioned to answer this one.

8

You can look at this script.