APPP – a Python Implementation of APJP

Important! Encryption in APJP and APPP (and by extension other proxy software that make use of these) is incorrectly implemented. Do NOT use these with vital private info, or at the very least use these over HTTPS.


Latest version is 0.1.3

 

What?

APJP is a proxy (originally) written in PHP and JAVA. It uses PHP/Python/JAVA application deployed on remote server as server-side, JAVA application installed on local computer/mobile device as client-side, and ARC4 encryption in-between.

APPP is a Python implementation of APJP. Currently, APPP only offers client-side application, but it should work with any APJP server-side application regardless of the language used by the latter.

Why?

There have been only JAVA-based client-side applications for Linux, Windows and Android. Some people may be troubled by the following issues:

  • JAVA is not available in their OS of choice, or not freely available, or not freely available under a ‘free’ license, or they are simply not happy about the dismaying JAVA vulnerabilities;
  • Memory footprint of JAVA applications precludes their use in embedded devices, such as routers;
  • It’s cumbersome to configure and run multiple JAVA-based APJP clients simultaneously.

APPP was intended to address these issues. Over time, new functionalities have also been added.

How?

For instructions on deployment of server-side APJP applications, please visit APJP wiki.

APPP requires Python 2.6 and above, and pyOpenSSL. These are available by default on most desktop and server distros of Linux. For Windows, either install Python and pyOpenSSL manually or use the precompiled Win32 application in the package linked below. APPP is compatible with Python3.

Download the APPP python script and packaged Win32 application here, define an APPP server in APPP.ini to match server-side deployment, and from within the appp python folder type ‘/path/to/python appp.py servername’ or for Win32 application, type ‘appp servername’ to start.

To disable certificate warnings prompted by browsers when visiting https sites through APPP, import ‘APPP.pem’ (generated after the first run) in appp folder into your browser as Trusted Root Certification Authorities (CA).

Tips and Tricks

Security versus Performance

APJP/APPP encrypts entire proxied traffic using the key shared by the client and server. As long as the key is not compromised (do NOT use a key that might be known to someone else!), it is very, very, very difficult for the proxied traffic to be decrypted if the connection is being eavesdropped on or under man-in-the-middle attacks. But that is not to say decryption is not possible. If this possibility really concerns you, one way to make the connection slightly more secure and decryption more difficult is to run APJP server over https to add an extra layer of protection.

Two considerations need to be addressed before going the https way. First, additional encryption and decryption introduced by https will cause extra CPU, memory and network usage, possibly resulting in slower performance. Second, a valid certificate is required for APJP over https to make actual sense security-wise. APPP checks certificate of APJP server running https against Mozilla’s root ca bundle and by default, refuses to connect if certificate is invalid. This behavior could be overridden in the config file, but the appropriate solution if a self-signed certificate need to be used is to add the self-generated root ca to the cacert.pem file.

Transparent Proxy

Not all internet-connecting applications support proxies, especially on mobile and embedded devices. The only way around this is to use DNS or firewall tricks to redirect the traffic to a transparent proxy server. APPP works as both normal proxy and transparent proxy, without any special setting. APPP could also work as a dedicated transparent https proxy if TRANSPARENT_HTTPS is set to yes. Here ‘dedicated’ means only normal https requests could be handled and all other types of requests will fail. Fortunately, it is most trivial to run multiple APPP proxies simultaneously in a single process to handle all types of http/https requests. Unfortunately, visiting https sites through transparent proxy will prompt invalid certificate warnings. To minimize the inconveniences, specify all frequently visited domains in the TRANSPARENT_HTTPS_DOMAINS setting.

Problems?

Append ‘-d’ option when starting APPP to get detailed debugging messages. Post the messages here if still at a loss.

Changelog

20130310: 0.1.3

Allow specifying alternative config file using ‘-c’ option;

Handle normal http requests (as transparent http proxy);

Add support for APJP server running over https;

Handle normal https requests (as transparent https proxy only).

20120519: 0.1.2

Add TIMEOUT configuration; better timeout handling.

20120519: 0.1.1

Fix compatibility with APJP_REMOTE_PHP > 0.8.2.

20120515: 0.1

Initial release.

DISCLAIMER

ABSOLUTELY NO WARRANTY. USE IT AT YOUR OWN RISK.

48 Responses to APPP – a Python Implementation of APJP

  1. Phil says:

    Hi, Farter, I have open on the hotfile link you gave, but only win32 app, no python script, I want to deploy it on my iOS device, so can you plz upload it again, thanks!

  2. Eric says:

    With APJP java version, all servers are listening to single port and APJP banlance among all the servers. for APPP, single server listens to a port. Is it possible to have banlance implementation in your great appp?

    • farter says:

      Yes it is possible, but I’m not gonna implement it.

      Having such a setup means your source IP, as website/servers see it, is constantly changing.

      This creates all sorts of problems.

      • Eric says:

        this will happen only with multi servers. but people who need unchanged ip can config only one server in the config file. even you can give a parameter to enable user config what sites they need unchanged ip and others thru banlanced servers.

        • farter says:

          I can not understand why anybody would want to use contantly changing IP. If you really want it, simply use batch or shell to restart APPP with different server every now and then.

  3. Pentium says:

    How to solve this problem: ERROR: Exception encountered forwarding request: u’/_URL
    Thanks.

    • farter says:

      Well, that description does not give any details at all. But my wild guess is you messed up the INI file.

      • Pentium says:

        Actually I run APPP on my router and try to use it for some website.
        I add this line “address=/.vimeo.com/.vimeocdn.com/111.111.111.111” to dnsmasq configure file.
        I also create a redirecting rule “iptables -t nat -I PREROUTING -i br0 -p tcp -d 111.111.111.111 –dport 80 -j DNAT –to-destination 192.168.1.1:8087”.
        The listen address and port is set like this “0.0.0.0:8087”.
        Then this error occurs. But when I use extensions like “switchsharp” in chrome to set proxy like this “192.168.1.1:8087”, it works.

  4. master says:

    What are the requirements for this script ?
    Whether there is a script to be installed on the host , and this requirement is to show up or not ?
    Free hosting that allows you to install it ?

  5. yourfans says:

    And APJP has been updated to 1.0.0. There should be something news in the latest version. Can you update the APPP to fix up some bugs and achieve the new function too?
    Thank you again and again ^_^

    • farter says:

      There is not much that is ‘new’ regarding the protocol, just some changes to implementation details, which are almost unnoticeble to users.

  6. yourfans says:

    There is a function in APJP’s properties file like this:
    APJP_REMOTE_HTTP_SERVER_1_REQUEST_URL=https://google.co.jp/HTTP
    APJP_REMOTE_HTTP_SERVER_1_REQUEST_PROPERTY_1_KEY=Host
    APJP_REMOTE_HTTP_SERVER_1_REQUEST_PROPERTY_1_VALUE=my_app_id.appspot.com
    Because mainland China can not open the appspot site directly,this property is useful to work on GAE.

    I try to set up the APPP.ini file like this:
    [EXAMPLE]
    HTTP_URL=https://google.co.jp/HTTP
    HTTPS_URL=https://google.co.jp/HTTPS
    CUSTOM_HEADER1=Host:my_app_id.appspot.com
    (or CUSTOM_HEADER1=Host: my_app_id.appspot.com)
    (or CUSTOM_HEADER1=’Host:my_app_id.appspot.com’)
    (or CUSTOM_HEADER1=’Host: my_app_id.appspot.com’)
    However,none is work. Even do not pass the HTTP_TEST_SITE/HTTPS_TEST_SITE.

    How can APPP to achieve the same functionality?

    • farter says:

      Hi,

      I’ve tested that using IP in HTTP(S)_URL and hostname in CUSTOM_HEADER works with shared hosting. So you can try using google IP instead of google.co.jp. You can also try using http://google.co.jp instead of the https.

      • yourfans says:

        HTTP_URL(S)=http://google_ip_in_china/HTTP(S)
        CUSTOM_HEADER1=Host:my_app_id.appspot.com
        It works. Thank you.

        • yourfans says:

          Because of the improvement of China GFW,above method requesting over HTTP has died no matter which IP.(request reset)
          APJP is still alive as it can use HTTPS.
          So,hope to implement the HTTPS request.
          HTTP_URL(S)=https://google_ip_in_china/HTTP(S)
          CUSTOM_HEADER1=Host:my_app_id.appspot.com

          • farter says:

            As I mentioned earlier, APJP/APPP over https wastes server resources unnecessarily. The easiest solution would be not using Google appspot, which is not meant to be used for such purposes anyway, and use other hosting/vps/server services not blocked by the wall.

  7. Liberty says:

    客户端appp,服务端java的PaaS,发现个别网站如http://boxun.com/无法访问,谷歌图片搜索中也有很多图片无法显示。是不是服务端屏蔽了部分网址?
    如果appp加上最小化到托盘的功能就完美了。
    Thanks

    • farter says:

      Hi, you can enable debugging, try to visit the problematic files or images, and see the log for clues.

      If you are using the appp.py script, you can use ‘pythonw’ instead of ‘python’ to execute the script. No cmd console will be shown, but you will miss all the logs.

      • Liberty says:

        我的意思是在Win32 application客户端中加个可以最小化到托盘的启动,就像goagent.exe那样的就很好。

  8. @twfcc says:

    this is the error message from JB iPad2 iOS 511 with python 2.7.1 & pyOpenSSL installed.

    “Address already in use” message is appeared no matter I change 127.0.0.1:10000-2
    Any idea? Thanks

    07:00:53 APPP[MainThread] INFO: Attempting to run all APPP servers defined in APPP.ini
    07:00:53 APPP[MainThread] INFO: Processing [EXAMPLE] configurations
    07:00:53 APPP[MainThread] INFO: Processing [EXAMPLE-PROXY] configurations
    07:00:53 APPP[MainThread] INFO: Checking APPP certificates…
    07:00:53 APPP[MainThread] INFO: APPP certificates OK.
    07:00:53 APPP[EXAMPLE-PROXY] INFO: Listening on 127.0.0.1:10001.
    07:00:53 APPP[EXAMPLE-PROXY-TEST] INFO: Testing APPP connectivity to http://www.google.com
    07:00:53 APPP[EXAMPLE] ERROR: Binding to 127.0.0.1:10001 failed: [Errno 48] Address already in use
    07:00:53 APPP[EXAMPLE-PROXY-TEST] INFO: Testing APPP connectivity to https://www.google.com
    07:00:53 APPP[EXAMPLE-PROXY-805195776] INFO: Received connection from 127.0.0.1:56680
    07:00:53 APPP[EXAMPLE-PROXY-18354176] INFO: Received connection from 127.0.0.1:56681
    07:00:54 APPP[EXAMPLE-PROXY-805195776] ERROR: Exception encountered processing request from 127.0.0.1:56680: [Errno 54] Connection reset by peer
    07:00:54 APPP[EXAMPLE-PROXY-18354176] ERROR: Exception encountered processing request from 127.0.0.1:56681: [Errno 54] Connection reset by peer
    07:00:54 APPP[EXAMPLE-PROXY-TEST] ERROR: APPP connectivity test to http://www.google.com failed: no correctly formatted response header received.
    07:00:54 APPP[EXAMPLE-PROXY-TEST] ERROR: APPP connectivity test to https://www.google.com failed: no correctly formatted response header received.

    • farter says:

      ‘Address already in use’ because you are running multiple APPP servers but failed to set them to bind to different ports.

      In your log, EXAMPLE-PROXY was the first to be run and bind successfully ’07:00:53 APPP[EXAMPLE-PROXY] INFO: Listening on 127.0.0.1:10001.’

      • littlewater says:

        The same error as twfcc.
        with enabled log-debug, I found the error may caused by nothing-data return in “client_buffer” at line 306(ver 0.1.2)
        And I tried using wireshark to trace the error As following :
        (where 192.168.1.101 is my local IP)

        No. Time Source Destination Protocol Length Info
        18 5.084760000 192.168.1.101 50.16.251.76 TCP 66 9688 > https [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=4 SACK_PERM=1
        19 5.086134000 192.168.1.101 50.16.251.76 TCP 66 9689 > https [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=4 SACK_PERM=1
        20 5.410802000 50.16.251.76 192.168.1.101 TCP 66 https > 9688 [SYN, ACK] Seq=0 Ack=1 Win=5840 Len=0 MSS=1452 SACK_PERM=1 WS=256
        21 5.410892000 192.168.1.101 50.16.251.76 TCP 54 9688 > https [ACK] Seq=1 Ack=1 Win=66792 Len=0
        22 5.412722000 192.168.1.101 50.16.251.76 SSL 258 Continuation Data
        23 5.412977000 50.16.251.76 192.168.1.101 TCP 66 https > 9689 [SYN, ACK] Seq=0 Ack=1 Win=5840 Len=0 MSS=1452 SACK_PERM=1 WS=256
        24 5.413022000 192.168.1.101 50.16.251.76 TCP 54 9689 > https [ACK] Seq=1 Ack=1 Win=66792 Len=0
        25 5.414534000 192.168.1.101 50.16.251.76 SSL 259 Continuation Data
        28 5.738494000 50.16.251.76 192.168.1.101 TCP 60 https > 9688 [ACK] Seq=1 Ack=205 Win=6912 Len=0
        29 5.738547000 192.168.1.101 50.16.251.76 SSL 114 Continuation Data
        30 5.739062000 50.16.251.76 192.168.1.101 TCP 60 https > 9688 [FIN, ACK] Seq=1 Ack=205 Win=6912 Len=0
        31 5.739095000 192.168.1.101 50.16.251.76 TCP 54 9688 > https [ACK] Seq=265 Ack=2 Win=66792 Len=0
        32 5.739566000 50.16.251.76 192.168.1.101 TCP 60 https > 9688 [RST, ACK] Seq=2 Ack=205 Win=6912 Len=0
        33 5.745162000 50.16.251.76 192.168.1.101 TCP 60 https > 9689 [ACK] Seq=1 Ack=206 Win=6912 Len=0
        34 5.745194000 192.168.1.101 50.16.251.76 SSL 114 Continuation Data
        35 5.745719000 50.16.251.76 192.168.1.101 TCP 60 https > 9689 [FIN, ACK] Seq=1 Ack=206 Win=6912 Len=0
        36 5.745745000 192.168.1.101 50.16.251.76 TCP 54 9689 > https [ACK] Seq=266 Ack=2 Win=66792 Len=0
        37 5.746231000 50.16.251.76 192.168.1.101 TCP 60 https > 9689 [RST, ACK] Seq=2 Ack=206 Win=6912 Len=0
        41 6.067161000 50.16.251.76 192.168.1.101 TCP 60 https > 9688 [RST] Seq=2 Win=0 Len=0
        42 6.073064000 50.16.251.76 192.168.1.101 TCP 60 https > 9689 [RST] Seq=1 Win=0 Len=0
        43 6.073477000 50.16.251.76 192.168.1.101 TCP 60 https > 9689 [RST] Seq=2 Win=0 Len=0

        the server and client without any hello exists, I thought it may failed during SSL validation??

        • littlewater says:

          well, I thought I got the key …
          I guess APPP won’t support SSL but should use http:// instead of default APJP https:// connection ….

          after changed this, the logger reported OK for everything, and even this post MAY post by APPP, but I doubt this method safety ….

          But compared with large costs JAVA, this is really memory saved for python though ….

          • farter says:

            No, appp over https is not implemented, as it seems to be a little bit redundant (encryption within encryption) and resource-wasteful (encryption overhead).

  9. niming says:

    system: centos6.2
    problem:outside the appp.py path,run ‘python /path/to/appp.py -a’ will not work, showing not found the appp.ini file.similar project goagent do not have the problem.
    http://code.google.com/p/goagent

    • farter says:

      Maybe next version. Or you could just use ‘cd /path/to/appp.py && python appp.py -a’.

      • niming says:

        i’ve created a sh file appp.sh with this code:
        cd /path/to/appp.py
        python appp.py -a
        and set to ‘startup application’.every time logined,it will run automatically.

  10. @twfcc says:

    great jobs, APJP now can running on JailBreak iOS device, thanks!

Comments are closed.