Update README

README.md

@@ -1,14 +1,16 @@
 # mtg
 
-Bullshit-free MTPROTO proxy for Telegram
+Highly-opionated (ex-bullshit-free) MTPROTO proxy for
+[Telegram](https://telegram.org/).
 
-[![Build Status](https://travis-ci.org/9seconds/mtg.svg?branch=master)](https://travis-ci.org/9seconds/mtg)
-[![Go Report Card](https://goreportcard.com/badge/github.com/9seconds/mtg)](https://goreportcard.com/report/github.com/9seconds/mtg)
-[![Docker Build Status](https://img.shields.io/docker/build/nineseconds/mtg.svg)](https://hub.docker.com/r/nineseconds/mtg/)
+[![CI](https://github.com/9seconds/mtg/actions/workflows/ci.yaml/badge.svg?branch=master)](https://github.com/9seconds/mtg/actions/workflows/ci.yaml)
+[![codecov](https://codecov.io/gh/9seconds/mtg/branch/master/graph/badge.svg?token=JfdDyGVpT4)](https://codecov.io/gh/9seconds/mtg)
+[![Go Reference](https://pkg.go.dev/badge/github.com/9seconds/mtg.svg)](https://pkg.go.dev/github.com/9seconds/mtg)
 
-**Please see a guide on upgrading to 1.0 at the end of this README.**
+**If you use v1.0 or upgrade broke you proxy, please read the chapter
+[Version 2](#version-2)**
 
-# Rationale
+## Rationale
 
 There are several available proxies for Telegram MTPROTO available. Here
 are the most notable:
@@ -17,267 +19,322 @@ are the most notable:
 * [Python](https://github.com/alexbers/mtprotoproxy)
 * [Erlang](https://github.com/seriyps/mtproto_proxy)
 
-Almost all of them follow the way how official proxy was built. This
-includes support of multiple secrets, support of promoted channels, etc.
+You can use any of these. They work great and all implementations have
+feature parity now. This includes support of adtag, replay attack
+protection, domain fronting, faketls, and so on. mtg has a similar
+goal: to give a possibility to connect to Telegram in a restricted,
+censored environment. But it does it slightly differently in details
+that probably matter.
 
-mtg is an implementation in golang which is intended to be:
+* **Resource-efficient**
+
+  It has to be resource-efficient. It does not mean that you will see
+  the smallest memory usage. It means that it will try to use allocated
+  resources in zero-waste mode, reusing as much memory as possible and
+  so on.
 
-* **Lightweight**
-  It has to consume as few resources as possible but not by losing
-  maintainability.
 * **Easily deployable**
+
   I strongly believe that Telegram proxies should follow the way of
-  ShadowSocks: promoted channels is a strange way of doing business
-  I suppose. I think the only viable way is to have a proxy with
-  minimum configuration which should work everywhere.
+  [ShadowSocks](https://shadowsocks.org): promoted channels is a strange
+  way of doing business I suppose. I think the only viable way is to
+  have a proxy that can be restored anywhere easily.
+
 * **A single secret**
-  I think that multiple secrets solve no problems and just complexify
+
+  I think that multiple secrets solve no problems and just complex
   software. I also believe that in the case of throwout proxies, this
-  feature is a useless luxury.
-* **Minimum docker image size**
-  Official image is less than 3.5 megabytes. Literally.
-* **No management WebUI**
-  This is an implementation of a simple lightweight proxy. I won't do that.
+  the feature is a useless luxury.
 
-This proxy supports 2 modes of work: direct connection to Telegram and
-promoted channel mode. If you do not need promoted channels, I would
-recommend you to go with direct mode: this way is more robust.
+* **No adtag support**
 
-To run a proxy in direct mode, all you need to do is just provide a
-secret. If you do not provide ADTag as a second parameter, promoted
-channels mode won't be activated.
+  Please read [Version 2](#version-2) chapter.
 
-To get promoted channel, please contact
-[@MTProxybot](https://t.me/MTProxybot) and provide generated adtag as a
-second parameter.
+* **No management WebUI**
 
+  This is an implementation of a simple lightweight proxy. I won't do that.
 
-# Source code organization
+* **Proxy chaining**
+
+  mtg has the support of [SOCKS5](https://en.wikipedia.org/wiki/SOCKS)
+  proxies. So, in theory, you can run this proxy as a frontend
+  and route traffic via [v2ray](https://www.v2ray.com/),
+  [Gost](https://docs.ginuerzh.xyz/gost/),
+  [Trojan](https://trojan-gfw.github.io/trojan/), or any other project
+  you like.
+
+* **Native blocklist support**
+
+  Previously, this was delegated to the [FireHOL](https://firehol.org/)
+  project or similar ones which track attacks and publish a list of
+  potentially dangerous IPs. mtg has native support of such blocklists.
+
+* **Can be used as a library**
 
-There are 2 main branches:
+  mtg v2 was redesigned in a way so it can be embedded into your
+  software (written in Golang) with a minimum effort + you can replace
+  some parts with those you want.
 
-1. `master` branch contains potentially unstable features
-2. `stable` branch contains stable version. Usually you want to use this branch.
+### Version 2
 
-# How to build
+If you use version 1.x before, you are probably noticed some major
+backward non-compatible details:
 
-```console
-make
-```
+1. Configuration file
+2. Removed support of adtag
 
-If you want to build for another platform:
+For the configuration file, please check out the full example in this
+repository. It has a lot of comments and most of the options are
+optional. We do have only `secret` and `bind-to` sections mandatory.
+Other sections in the example configuration file are filled with default
+values.
 
-```console
-make crosscompile
-```
+Adtag support was removed completely. This was done to debloat mtg and
+keep it simple and obvious. Hopefully, this goal is achieved and the
+source code is clean and straightforward enough.
 
-If you want to build Docker image (called `mtg`):
+I always was quite skeptical about adtag. In my POV, a proxy as a fat
+big connectivity point for hundreds of clients is an illusion. If you
+work in a censored environment, the first thing that authority does is
+IP blocking. For us, it means, those big proxies that can benefit from
+having a pinned channel are going to be blocked in a minute.
 
-```console
-make docker
-```
+Proxy has to be intimate. It has to be shared within a small group as
+a family or maybe your college friends. It has to have a small number
+of connections and never publicly announced its presence. It has to fly
+under the radar. If the proxy is detected, you need to be able to give
+a rebirth on a new IP address as soon as possible. I do no think that
+having some special channel for such a use case makes any sense.
 
-# Docker image
+But other details like replay attack protection, domain fronting,
+accurate FakeTLS implementation, IP blacklisting, and proxy
+chaining matter here. If you work in censored perimeter like
+[GFW](https://en.wikipedia.org/wiki/Great_Firewall)-protected
+country, you probably want to have an MTPROTO proxy as
+a frontend that transports traffic via cloaked tunnels
+made by [Trojan](https://trojan-gfw.github.io/trojan/),
+[Shadowsocks](https://shadowsocks.org), [v2ray](https://www.v2ray.com/),
+or [Gost](https://docs.ginuerzh.xyz/gost/). That's why you have to have
+the support of chaining as a first-class citizen.
 
-Docker follows the same policy as the source code organization:
+Yes, this is possible and doable with optional adtag support. But the
+truth is that the MTPROTO proxy for Telegram is just a thing that either
+work as a normal client (direct mode) or doing some RPC calls in [TL
+language](https://core.telegram.org/mtproto/TL) (adtag support). I
+understand the intention of the developers and I understand that they
+were under high pressure fighting with [RKN](https://rkn.gov.ru/) and
+doing TON after that. Nothing is ideal. But for the proxy, it means that
+source code is full of complex non-trivial code which is required only
+to support a feature that we barely need.
 
-- `latest` mirrors the master branch
-- `stable` mirrors the stable branch
-- tags are for tagged releases
+So, to have a reasonable MTPROTO proxy, adtag support was removed. This
+is a rare chance in my career where software v2 debloats a previous
+version. It feels so good :)
 
-```console
-docker pull nineseconds/mtg:latest
-```
+### Version 1 and 2
 
-```console
-docker pull nineseconds/mtg:stable
-```
+I do continue to support both versions 1 and 2. But in a different mode.
 
-```console
-docker pull nineseconds/mtg:0.10
-```
+Version 1 is now officially in maintenance mode. It means that I won't
+make any new features or improvements there. You can consider a feature
+freeze there. No bugs are going to be fixed there except for critical
+ones. PRs are welcome though. The goal is to keep it working. It will
+get some periodical updates like updates to the new Golang version of
+dependencies version bump, but that's mostly it.
 
-# Ansible role
+**If you want to have mtg with _adtag support_, please use version 1**.
 
-You can find unofficial Ansible role for mtg here: https://github.com/rlex/ansible-role-mtg
-Also, there is another project on Ansible Galaxy: https://galaxy.ansible.com/ivansible/lin_mtproxy
+Version 2 is going to have all my love, active support, bug fixing, etc.
+It is under active development and maintenance.
 
-# Configuration
+This project has several main branches
 
-To run this tool you need to configure as less as possible. Telegram
-clients support 3 different secret types:
+1. [`master`](https://github.com/9seconds/mtg/tree/master) branch
+   contains a bleeding edge. It may potentially have some features
+   which will break your source code.
+2. [`stable`](https://github.com/9seconds/mtg/tree/stable) branch contains
+   dumps of a master branch when we consider it 'stable'. This is a
+   branch you probably want to pick.
+3. [`v2`](https://github.com/9seconds/mtg/tree/v2) has a development
+   of the v2.x version. In theory, it is the same as `master` but this
+   will change when we have v3.x.
+4. [`v1`](https://github.com/9seconds/mtg/tree/v1) has a version 1.x.
+
+## Getting started
 
-* Simple - basically, it is just a flow of frames ciphered by AES-CTR stream
-  cipher.
-* Secured - the same stream as simple but with some random noise to prevent
-  statistical analysis of traffic flow.
-* FakeTLS - this mode envelops telegram stream in TLS so it looks (in theory)
-  the same as any TLS1.3 traffic from DPI point of view.
+### Download a tool
+
+#### Download binaries
+
+Binaries can be downloaded from the release page. Also, you can download
+docker image.
 
-If you do not have preferences, go with FakeTLS or at least secured.
-Simple mode is a little bit naive and traffic flow can be easily
-identified as Telegram one.
+For the current version, please download like
 
-Unlike the rest of implementation, mtg is quite strict about the
-execution mode: if you run a proxy instance with FakeTLS secret, you
-can't connect to it with simple or secured clients. You can't connect
-to the proxy with secured secret with FakeTLS key. It forces one mode
-of working. So, unfortunately, there is no way how to connect to the
-deployed proxy with another secret (if you know how to construct and
-convert them). But at the same time, old clients can't connect so they
-won't expose the type of the service.
+```console
+docker pull nineseconds/mtg:2
+```
 
-First, you need to generate a secret:
+For version 1:
 
 ```console
-$ mtg generate-secret simple
-52a493bdfb90eea55739eabff2d92a14
+docker pull nineseconds/mtg:1
 ```
 
+You may also check both [Docker
+Hub](https://hub.docker.com/r/nineseconds/mtg/tags) and [Github
+Registry](https://github.com/users/9seconds/packages/container/package/mtg).
+Please do not choose `latest` or `stable` if you want to avoid
+surprises. Always choose some version tag.
+
+Also, if you have `go` installed, you can always download this tool with `go get`:
+
 ```console
-$ mtg generate-secret secured
-ddf05fb7acb549be047a7c585116581418
+go get github.com/9seconds/mtg/v2
 ```
 
+#### Build from sources
+
 ```console
-$ mtg generate-secret -c google.com tls
-ee852380f362a09343efb4690c4e17862e676f6f676c652e636f6d
+git clone https://github.com:9seconds/mtg.git
+cd mtg
+make static
 ```
 
-Or, if you prefer docker:
+or for the docker image:
 
 ```console
-$ docker run --rm nineseconds/mtg generate-secret tls -c bing.com
-eedf71035a8ed48a623d8e83e66aec4d0562696e672e636f6d
+make docker
 ```
 
-## Antireplay cache
+### Generate secret
 
-To prevent replay attacks, we have internal storage of first frames
-messages for connected clients. These frames are generated randomly
-by design and we have the negligible possibility of duplication
-(probability is 1/(2^64)) but it could be quite effective to prevent
-replays.
+If you already have a secret in Base64 format or that, which starts with `ee`,
+you can skip this chapter. Otherwise:
 
-It is possible to disable this cache. To do that, please explicitly set
-its size to 0.
+```console
+$ mtg generate-secret google.com
+7ibaERuTSGPH1RdztfYnN4tnb29nbGUuY29t
+```
 
+or
 
-## FakeTLS
+```console
+$ mtg generate-secret --hex google.com
+ee473ce5d4958eb5f968c87680a23854a0676f6f676c652e636f6d
+```
 
-If you run this a proxy in faketls mode, this proxy will try to hide
-itself cloaking a host provided as a part of the generated secret. It
-means that if you cloak google.com then you can curl this proxy and
-you'll get a google.com response back.
+This secret is a keystone for a proxy and your password for a client.
+You need to keep it secured.
 
-mtg proxies L3 traffic. In other words, only TCP, without interfering in
-TLS, HTTP or any other high-level protocol.
+We recommend choosing a hostname wisely. Here we have a _google.com_
+but in reality, all providers can easily detect that this is not a
+Google. Google has a list of networks it officially uses and your IP
+address won't probably belong to it. It is a great idea to hide behind
+some domain that has some relation to this IP address.
 
+For example, you've bought a VPS from [Digital
+Ocean](https://www.digitalocean.com/). Then it might be a good idea to
+generate a secret for _digitalocean.com_ then.
 
-## Environment variables
+### Prepare a configuration file
 
-It is possible to configure this tool using environment variables. You
-can configure any flag but not secret or adtag. Here is the list of
-supported environment variables:
+Please checkout an example configuration file. All options except of
+`secret` and `bind-to` are optional. You can safely have this minimal
+configuration file:
 
-| Environment variable          | Corresponding flags          | Default value                     | Description                                                                                                                                                                                                                                                                     |
-|-------------------------------|------------------------------|-----------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `MTG_DEBUG`                   | `-d`, `--debug`              | `false`                           | Run in debug mode. Usually, you need to run in this mode  only if you develop this tool or its maintainer is asking you to provide  logs with such verbosity.                                                                                                                   |
-| `MTG_VERBOSE`                 | `-v`, `--verbose`            | `false`                           | Run in verbose mode. This is way less chatty than debug mode.                                                                                                                                                                                                                   |
-| `MTG_BIND`                    | `-b`, `--bind`               | `0.0.0.0:3128`                    | Which host/port pair should we bind to (listen on).                                                                                                                                                                                                                             |
-| `MTG_IPV4`                    | `-4`, `--public-ipv4`        | [Autodetect](https://ifconfig.co) | IPv4 address:port of this proxy. This is required if you NAT your proxy or run it in a docker container. In that case, you absolutely need to specify public IPv4 address of the proxy, otherwise either URLs will be broken or proxy could not access Telegram middle proxies. |
-| `MTG_IPV6`                    | `-6`, `--public-ipv6`        | [Autodetect](https://ifconfig.co) | IPv6 address:port of this proxy. This is required if you NAT your proxy or run it in a docker container. In that case, you absolutely need to specify public IPv6 address of the proxy, otherwise either URLs will be broken or proxy could not access Telegram middle proxies. |
-| `MTG_STATS_BIND`              | `-t`, `--stats-bind`         | `127.0.0.1:3129`                  | Which hist:port should we bind the internal statistics HTTP server (Prometheus).                                                                                                                                                                                                |
-| `MTG_STATS_NAMESPACE`         | `--stats-namespace`          | `mtg`                             | Which namespace should be used for prometheus metrics.                                                                                                                                                                                                                          |
-| `MTG_STATSD_ADDR`             | `--statsd-addr`              |                                   | host:port of statsd service. No defaults, by default we do not send anything there.                                                                                                                                                                                             |
-| `MTG_STATSD_PREFIX`           | `--statsd-prefix`            | `mtg`                             | Which bucket prefix we should use. For example, if you set `mtg`, then metric `traffic.ingress` would be send as `mtg.traffic.ingress`.                                                                                                                                         |
-| `MTG_STATSD_TAGS_FORMAT`      | `--statsd-tags-format`       |                                   | Which tags format we should use. By default, we are using default vanilla statsd tags format but if you want to send directly to InfluxDB or Datadog, please specify it there. Possible options are `influxdb` and `datadog`.                                                   |
-| `MTG_STATSD_TAGS`             | `--statsd-tags`              |                                   | Which tags should we send to statsd with our metrics. Please specify them as `key=value` pairs.                                                                                                                                                                                 |
-| `MTG_BUFFER_WRITE`            | `-w`, `--write-buffer`       | `32KB`                            | The size of TCP write buffer in bytes. Write buffer is the buffer for messages which are going from client to Telegram.                                                                                                                                                         |
-| `MTG_BUFFER_READ`             | `-r`, `--read-buffer`        | `32KB`                            | The size of TCP read buffer in bytes. Read buffer is the buffer for messages from Telegram to client.                                                                                                                                                                           |
-| `MTG_ANTIREPLAY_MAXSIZE`      | `--anti-replay-max-size`     | `128MB`                           | Max size of antireplay cache.                                                                                                                                                                                                                                                   |
-| `MTG_CLOAK_PORT`              | `--cloak-port`               | `443`                             | Which port we should use to connect to cloaked host in FakeTLS mode.                                                                                                                                                                                                            |
-| `MTG_MULTIPLEX_PERCONNECTION` | `--multiplex-per-connection` | `50`                              | How many client connections can share a single Telegram connection in adtag mode                                                                                                                                                                                                |
-| `MTG_NTP_SERVERS`             | `--ntp-server`               | default pool                      | A list of NTP servers to use.                                                                                                                                                                                                                                                   |
-| `MTG_PREFER_DIRECT_IP`        | `--prefer-ip`                | `ipv6`                            | Which IP protocol to prefer if possible. Works mostly in direct mode.                                                                                                                                                                                                           |
+```toml
+secret = "ee473ce5d4958eb5f968c87680a23854a0676f6f676c652e636f6d"
+bind-to = "0.0.0.0:443"
+```
 
+This is enough to run the whole application. All other
+options already have sensible defaults for the app at almost any scale.
 
-Usually you want to modify only read/write buffer sizes. If you feel
-that proxy is slow, try to increase both sizes giving more priority to
-read buffer.
+Oh, the configuration is done in [TOML format](https://toml.io/en/).
 
-Unfortunately, MTPROTO proxy protocol does not allow us to use splice
-or any other neat tricks how to eliminate the need of copying data into
-userspace.
+### Run a proxy
 
-# How to run the tool
+Put a binary and a config into your webserver. Just for example,
+a binary goes to `/usr/local/bin/mtg` and configuration to `/etc/mtg.toml`.
 
-Now run the tool:
+Now you can create a systemd unit:
 
 ```console
-$ mtg run <secret>
+$ cat /etc/systemd/system/mtg.service
+[Unit]
+Description=mtg
+
+[Service]
+ExecStart=/usr/local/bin/mtg run /etc/mtg.toml
+Restart=always
+RestartSec=3
+
+[Install]
+WantedBy=multi-user.target
+$ sudo systemctl daemon-reload
+$ sudo systemctl enable mtg
+$ sudo systemctl start mtg
 ```
 
-How to run the tool with ADTag:
+or you can run a docker image
 
 ```console
-$ mtg run <secret> <adtag>
+docker run -d -v /etc/mtg.toml:/config.toml -p 443:3128 --restart=unless-stopped nineseconds/mtg:2
 ```
 
-This tool will listen on port 3128 by default with the given secret.
+where _443_ is a host port (a port you want to connect to from a
+client), and _3128_ is the one you have in your config in the `bind-to`
+section.
 
+### Access a proxy
 
-# oneliner to run this proxy
-
-Please ensure that docker is installed. After that just execute
+Now you can generate some useful links:
 
 ```console
-curl -sfL --compressed https://raw.githubusercontent.com/9seconds/mtg/master/run.sh | bash
+$ mtg access /etc/mtg.toml
+{
+  "ipv4": {
+    "ip": "x.y.z.a",
+    "port": 3128,
+    "tg_url": "tg://proxy?...",
+    "tg_qrcode": "https://api.qrserver.com/v1/create-qr-code?data...",
+    "tme_url": "https://t.me/proxy?...",
+    "tme_qrcode": "https://api.qrserver.com/v1/create-qr-code?data..."
+  },
+  "secret": {
+    "hex": "...",
+    "base64": "..."
+  }
+}
 ```
 
-
-# statsd integration
-
-mtg provides an integration with statsd, you can enable it with command
-line interface. To enable it, you have to provide IP address of statsd
-service.
-
-Out of the box, mtg supports 2 additional dialects: [InfluxDB](https://www.influxdata.com/blog/getting-started-with-sending-statsd-metrics-to-telegraf-influxdb/)
-and [Datadog](https://docs.datadoghq.com/developers/dogstatsd/).
-
-All metrics are gauges. Here is the list of metrics and their meaning:
-
-| Metric name            | Unit    | Description                                |
-|------------------------|---------|--------------------------------------------|
-| `connections`          | number  | The number of active connections.          |
-| `telegram_connections` | number  | The number of active telegram connections. |
-| `crashes`              | number  | An amount of crashes in client handlers.   |
-| `traffic.egress`       | bytes   | Traffic from the start of application.     |
-| `replay_attacks`       | number  | The number of prevented replay attacks.    |
-
-All metrics are prefixed with given prefix. Default prefix is `mtg`.
-Also, metrics provide tags (ipv4/ipv6, dc indexes etc).
-
-
-# Prometheus integration
-
-[Prometheus](https://prometheus.io) integration comes out of
-the box, you do not need to setup anything special.
-
-
-# Upgrade to 1.0
-
-Version 1.0 breaks compatibility with previous versions so please read
-this chapter carefully:
-
-1. mtg now uses subcommands. Please use `mtg run` instead of just
-   `mtg` to run a proxy.
-2. Options which set host and port separately were removed in a
-   favor of fused `host:port` options.
-3. Own stats server was removed. Prometheus endpoint is moved to
-   default stats endpoint.
-4. It is possible to connect to this proxy only with a secret which
-   was used to run it. So, no backward compatibility of clients.
-5. Multiplexing involves connectivity with middle proxies and involves
-   the most complex code path of this proxy. To avoid potential bugs,
-   we still recommend using direct mode.
+## Metrics
+
+Out of the box, mtg works with
+[statsd](https://github.com/statsd/statsd) and
+[Prometheus](https://prometheus.io/). Please check configuration file
+example to get how to set this integration up.
+
+Here goes a list of metrics with their types but without a prefix.
+
+| Name                        | Type    | Tags                             | Description                                                                                |
+|-----------------------------|---------|----------------------------------|--------------------------------------------------------------------------------------------|
+| client_connections          | gauge   | `ip_family`                      | Count of processing client connections.                                                    |
+| telegram_connections        | gauge   | `telegram_ip`, `dc`              | Count of connections to Telegram servers.                                                  |
+| domain_fronting_connections | gauge   | `ip_family`                      | Count of connections to fronting domain.                                                   |
+| telegram_traffic            | counter | `telegram_ip`, `dc`, `direction` | Count of bytes, transmitted to/from Telegram.                                              |
+| domain_fronting_traffic     | counter | `direction`                      | Count of bytes, transmitted to/from fronting domain.                                       |
+| domain_fronting             | counter | –                                | Count of domain fronting events.                                                           |
+| concurrency_limited         | counter | –                                | Count of events, when client connection was rejected due to concurrency limit.             |
+| ip_blocklisted              | counter | –                                | Count of events when client connection was rejected because IP was found in the blacklist. |
+| replay_attacks              | counter | –                                | Count of detected replay attacks.                                                          |
+
+Tag meaning:
+
+| Name        | Values                 | Description                                   |
+|-------------|------------------------|-----------------------------------------------|
+| ip_family   | ipv4, ipv6             | A version of the IP protocol.                 |
+| dc          |                        | A number of the Telegram DC for a connection. |
+| telegram_ip |                        | IP address of the Telegram server.            |
+| direction   | to_client, from_client | A direction of the traffic flow.              |