'put'에 해당되는 글 3건

  1. 2013.08.27 Using cURL to automate HTTP jobs
  2. 2010.02.10 The HTTP verb PUT under Apache: Safe or Dangerous?
  3. 2009.08.19 Secure File Upload Check List With PHP
2013.08.27 18:46

Using cURL to automate HTTP jobs

Date:    Jan 19, 2011
 
                The Art Of Scripting HTTP Requests Using Curl
                =============================================
 
 This document will assume that you're familiar with HTML and general
 networking.
 
 The possibility to write scripts is essential to make a good computer
 system. Unix' capability to be extended by shell scripts and various tools to
 run various automated commands and scripts is one reason why it has succeeded
 so well.
 
 The increasing amount of applications moving to the web has made "HTTP
 Scripting" more frequently requested and wanted. To be able to automatically
 extract information from the web, to fake users, to post or upload data to
 web servers are all important tasks today.
 
 Curl is a command line tool for doing all sorts of URL manipulations and
 transfers, but this particular document will focus on how to use it when
 doing HTTP requests for fun and profit. I'll assume that you know how to
 invoke 'curl --help' or 'curl --manual' to get basic information about it.
 
 Curl is not written to do everything for you. It makes the requests, it gets
 the data, it sends data and it retrieves the information. You probably need
 to glue everything together using some kind of script language or repeated
 manual invokes.
 
1. The HTTP Protocol
 
 HTTP is the protocol used to fetch data from web servers. It is a very simple
 protocol that is built upon TCP/IP. The protocol also allows information to
 get sent to the server from the client using a few different methods, as will
 be shown here.
 
 HTTP is plain ASCII text lines being sent by the client to a server to
 request a particular action, and then the server replies a few text lines
 before the actual requested content is sent to the client.
 
 The client, curl, sends a HTTP request. The request contains a method (like
 GET, POST, HEAD etc), a number of request headers and sometimes a request
 body. The HTTP server responds with a status line (indicating if things went
 well), response headers and most often also a response body. The "body" part
 is the plain data you requested, like the actual HTML or the image etc.
 
 1.1 See the Protocol
 
  Using curl's option --verbose (-v as a short option) will display what kind
  of commands curl sends to the server, as well as a few other informational
  texts.
 
  --verbose is the single most useful option when it comes to debug or even
  understand the curl<->server interaction.
 
  Sometimes even --verbose is not enough. Then --trace and --trace-ascii offer
  even more details as they show EVERYTHING curl sends and receives. Use it
  like this:
 
      curl --trace-ascii debugdump.txt http://www.example.com/
 
2. URL
 
 The Uniform Resource Locator format is how you specify the address of a
 particular resource on the Internet. You know these, you've seen URLs like
 http://curl.haxx.se or https://yourbank.com a million times.
 
3. GET a page
 
 The simplest and most common request/operation made using HTTP is to get a
 URL. The URL could itself refer to a web page, an image or a file. The client
 issues a GET request to the server and receives the document it asked for.
 If you issue the command line
 
        curl http://curl.haxx.se
 
 you get a web page returned in your terminal window. The entire HTML document
 that that URL holds.
 
 All HTTP replies contain a set of response headers that are normally hidden,
 use curl's --include (-i) option to display them as well as the rest of the
 document. You can also ask the remote server for ONLY the headers by using
 the --head (-I) option (which will make curl issue a HEAD request).
 
4. Forms
 
 Forms are the general way a web site can present a HTML page with fields for
 the user to enter data in, and then press some kind of 'OK' or 'submit'
 button to get that data sent to the server. The server then typically uses
 the posted data to decide how to act. Like using the entered words to search
 in a database, or to add the info in a bug track system, display the entered
 address on a map or using the info as a login-prompt verifying that the user
 is allowed to see what it is about to see.
 
 Of course there has to be some kind of program in the server end to receive
 the data you send. You cannot just invent something out of the air.
 
 4.1 GET
 
  A GET-form uses the method GET, as specified in HTML like:
 
        <form method="GET" action="junk.cgi">
          <input type=text name="birthyear">
          <input type=submit name=press value="OK">
        </form>
 
  In your favorite browser, this form will appear with a text box to fill in
  and a press-button labeled "OK". If you fill in '1905' and press the OK
  button, your browser will then create a new URL to get for you. The URL will
  get "junk.cgi?birthyear=1905&press=OK" appended to the path part of the
  previous URL.
 
  If the original form was seen on the page "www.hotmail.com/when/birth.html",
  the second page you'll get will become
  "www.hotmail.com/when/junk.cgi?birthyear=1905&press=OK".
 
  Most search engines work this way.
 
  To make curl do the GET form post for you, just enter the expected created
  URL:
 
        curl "http://www.hotmail.com/when/junk.cgi?birthyear=1905&press=OK"
 
 4.2 POST
 
  The GET method makes all input field names get displayed in the URL field of
  your browser. That's generally a good thing when you want to be able to
  bookmark that page with your given data, but it is an obvious disadvantage
  if you entered secret information in one of the fields or if there are a
  large amount of fields creating a very long and unreadable URL.
 
  The HTTP protocol then offers the POST method. This way the client sends the
  data separated from the URL and thus you won't see any of it in the URL
  address field.
 
  The form would look very similar to the previous one:
 
        <form method="POST" action="junk.cgi">
          <input type=text name="birthyear">
          <input type=submit name=press value=" OK ">
        </form>
 
  And to use curl to post this form with the same data filled in as before, we
  could do it like:
 
        curl --data "birthyear=1905&press=%20OK%20"         http://www.example.com/when.cgi
 
  This kind of POST will use the Content-Type
  application/x-www-form-urlencoded and is the most widely used POST kind.
 
  The data you send to the server MUST already be properly encoded, curl will
  not do that for you. For example, if you want the data to contain a space,
  you need to replace that space with %20 etc. Failing to comply with this
  will most likely cause your data to be received wrongly and messed up.
 
  Recent curl versions can in fact url-encode POST data for you, like this:
 
        curl --data-urlencode "name=I am Daniel" http://www.example.com
 
 4.3 File Upload POST
 
  Back in late 1995 they defined an additional way to post data over HTTP. It
  is documented in the RFC 1867, why this method sometimes is referred to as
  RFC1867-posting.
 
  This method is mainly designed to better support file uploads. A form that
  allows a user to upload a file could be written like this in HTML:
 
    <form method="POST" enctype='multipart/form-data' action="upload.cgi">
      <input type=file name=upload>
      <input type=submit name=press value="OK">
    </form>
 
  This clearly shows that the Content-Type about to be sent is
  multipart/form-data.
 
  To post to a form like this with curl, you enter a command line like:
 
        curl --form upload=@localfilename --form press=OK [URL]
 
 4.4 Hidden Fields
 
  A very common way for HTML based application to pass state information
  between pages is to add hidden fields to the forms. Hidden fields are
  already filled in, they aren't displayed to the user and they get passed
  along just as all the other fields.
 
  A similar example form with one visible field, one hidden field and one
  submit button could look like:
 
    <form method="POST" action="foobar.cgi">
      <input type=text name="birthyear">
      <input type=hidden name="person" value="daniel">
      <input type=submit name="press" value="OK">
    </form>
 
  To post this with curl, you won't have to think about if the fields are
  hidden or not. To curl they're all the same:
 
        curl --data "birthyear=1905&press=OK&person=daniel" [URL]
 
 4.5 Figure Out What A POST Looks Like
 
  When you're about fill in a form and send to a server by using curl instead
  of a browser, you're of course very interested in sending a POST exactly the
  way your browser does.
 
  An easy way to get to see this, is to save the HTML page with the form on
  your local disk, modify the 'method' to a GET, and press the submit button
  (you could also change the action URL if you want to).
 
  You will then clearly see the data get appended to the URL, separated with a
  '?'-letter as GET forms are supposed to.
 
5. PUT
 
 The perhaps best way to upload data to a HTTP server is to use PUT. Then
 again, this of course requires that someone put a program or script on the
 server end that knows how to receive a HTTP PUT stream.
 
 Put a file to a HTTP server with curl:
 
        curl --upload-file uploadfile http://www.example.com/receive.cgi
 
6. HTTP Authentication
 
 HTTP Authentication is the ability to tell the server your username and
 password so that it can verify that you're allowed to do the request you're
 doing. The Basic authentication used in HTTP (which is the type curl uses by
 default) is *plain* *text* based, which means it sends username and password
 only slightly obfuscated, but still fully readable by anyone that sniffs on
 the network between you and the remote server.
 
 To tell curl to use a user and password for authentication:
 
        curl --user name:password http://www.example.com
 
 The site might require a different authentication method (check the headers
 returned by the server), and then --ntlm, --digest, --negotiate or even
 --anyauth might be options that suit you.
 
 Sometimes your HTTP access is only available through the use of a HTTP
 proxy. This seems to be especially common at various companies. A HTTP proxy
 may require its own user and password to allow the client to get through to
 the Internet. To specify those with curl, run something like:
 
        curl --proxy-user proxyuser:proxypassword curl.haxx.se
 
 If your proxy requires the authentication to be done using the NTLM method,
 use --proxy-ntlm, if it requires Digest use --proxy-digest.
 
 If you use any one these user+password options but leave out the password
 part, curl will prompt for the password interactively.
 
 Do note that when a program is run, its parameters might be possible to see
 when listing the running processes of the system. Thus, other users may be
 able to watch your passwords if you pass them as plain command line
 options. There are ways to circumvent this.
 
 It is worth noting that while this is how HTTP Authentication works, very
 many web sites will not use this concept when they provide logins etc. See
 the Web Login chapter further below for more details on that.
 
7. Referer
 
 A HTTP request may include a 'referer' field (yes it is misspelled), which
 can be used to tell from which URL the client got to this particular
 resource. Some programs/scripts check the referer field of requests to verify
 that this wasn't arriving from an external site or an unknown page. While
 this is a stupid way to check something so easily forged, many scripts still
 do it. Using curl, you can put anything you want in the referer-field and
 thus more easily be able to fool the server into serving your request.
 
 Use curl to set the referer field with:
 
        curl --referer http://www.example.come http://www.example.com
 
8. User Agent
 
 Very similar to the referer field, all HTTP requests may set the User-Agent
 field. It names what user agent (client) that is being used. Many
 applications use this information to decide how to display pages. Silly web
 programmers try to make different pages for users of different browsers to
 make them look the best possible for their particular browsers. They usually
 also do different kinds of javascript, vbscript etc.
 
 At times, you will see that getting a page with curl will not return the same
 page that you see when getting the page with your browser. Then you know it
 is time to set the User Agent field to fool the server into thinking you're
 one of those browsers.
 
 To make curl look like Internet Explorer 5 on a Windows 2000 box:
 
  curl --user-agent "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)" [URL]
 
 Or why not look like you're using Netscape 4.73 on an old Linux box:
 
  curl --user-agent "Mozilla/4.73 [en] (X11; U; Linux 2.2.15 i686)" [URL]
 
9. Redirects
 
 When a resource is requested from a server, the reply from the server may
 include a hint about where the browser should go next to find this page, or a
 new page keeping newly generated output. The header that tells the browser
 to redirect is Location:.
 
 Curl does not follow Location: headers by default, but will simply display
 such pages in the same manner it display all HTTP replies. It does however
 feature an option that will make it attempt to follow the Location: pointers.
 
 To tell curl to follow a Location:
 
        curl --location http://www.example.com
 
 If you use curl to POST to a site that immediately redirects you to another
 page, you can safely use --location (-L) and --data/--form together. Curl will
 only use POST in the first request, and then revert to GET in the following
 operations.
 
10. Cookies
 
 The way the web browsers do "client side state control" is by using
 cookies. Cookies are just names with associated contents. The cookies are
 sent to the client by the server. The server tells the client for what path
 and host name it wants the cookie sent back, and it also sends an expiration
 date and a few more properties.
 
 When a client communicates with a server with a name and path as previously
 specified in a received cookie, the client sends back the cookies and their
 contents to the server, unless of course they are expired.
 
 Many applications and servers use this method to connect a series of requests
 into a single logical session. To be able to use curl in such occasions, we
 must be able to record and send back cookies the way the web application
 expects them. The same way browsers deal with them.
 
 The simplest way to send a few cookies to the server when getting a page with
 curl is to add them on the command line like:
 
        curl --cookie "name=Daniel" http://www.example.com
 
 Cookies are sent as common HTTP headers. This is practical as it allows curl
 to record cookies simply by recording headers. Record cookies with curl by
 using the --dump-header (-D) option like:
 
        curl --dump-header headers_and_cookies http://www.example.com
 
 (Take note that the --cookie-jar option described below is a better way to
 store cookies.)
 
 Curl has a full blown cookie parsing engine built-in that comes to use if you
 want to reconnect to a server and use cookies that were stored from a
 previous connection (or handicrafted manually to fool the server into
 believing you had a previous connection). To use previously stored cookies,
 you run curl like:
 
        curl --cookie stored_cookies_in_file http://www.example.com
 
 Curl's "cookie engine" gets enabled when you use the --cookie option. If you
 only want curl to understand received cookies, use --cookie with a file that
 doesn't exist. Example, if you want to let curl understand cookies from a
 page and follow a location (and thus possibly send back cookies it received),
 you can invoke it like:
 
        curl --cookie nada --location http://www.example.com
 
 Curl has the ability to read and write cookie files that use the same file
 format that Netscape and Mozilla do. It is a convenient way to share cookies
 between browsers and automatic scripts. The --cookie (-b) switch
 automatically detects if a given file is such a cookie file and parses it,
 and by using the --cookie-jar (-c) option you'll make curl write a new cookie
 file at the end of an operation:
 
        curl --cookie cookies.txt --cookie-jar newcookies.txt         http://www.example.com
 
11. HTTPS
 
 There are a few ways to do secure HTTP transfers. The by far most common
 protocol for doing this is what is generally known as HTTPS, HTTP over
 SSL. SSL encrypts all the data that is sent and received over the network and
 thus makes it harder for attackers to spy on sensitive information.
 
 SSL (or TLS as the latest version of the standard is called) offers a
 truckload of advanced features to allow all those encryptions and key
 infrastructure mechanisms encrypted HTTP requires.
 
 Curl supports encrypted fetches thanks to the freely available OpenSSL
 libraries. To get a page from a HTTPS server, simply run curl like:
 
        curl https://secure.example.com
 
 11.1 Certificates
 
  In the HTTPS world, you use certificates to validate that you are the one
  you claim to be, as an addition to normal passwords. Curl supports client-
  side certificates. All certificates are locked with a pass phrase, which you
  need to enter before the certificate can be used by curl. The pass phrase
  can be specified on the command line or if not, entered interactively when
  curl queries for it. Use a certificate with curl on a HTTPS server like:
 
        curl --cert mycert.pem https://secure.example.com
 
  curl also tries to verify that the server is who it claims to be, by
  verifying the server's certificate against a locally stored CA cert
  bundle. Failing the verification will cause curl to deny the connection. You
  must then use --insecure (-k) in case you want to tell curl to ignore that
  the server can't be verified.
 
  More about server certificate verification and ca cert bundles can be read
  in the SSLCERTS document, available online here:
 
        http://curl.haxx.se/docs/sslcerts.html
 
12. Custom Request Elements
 
 Doing fancy stuff, you may need to add or change elements of a single curl
 request.
 
 For example, you can change the POST request to a PROPFIND and send the data
 as "Content-Type: text/xml" (instead of the default Content-Type) like this:
 
         curl --data "<xml>" --header "Content-Type: text/xml"               --request PROPFIND url.com
 
 You can delete a default header by providing one without content. Like you
 can ruin the request by chopping off the Host: header:
 
        curl --header "Host:" http://www.example.com
 
 You can add headers the same way. Your server may want a "Destination:"
 header, and you can add it:
 
        curl --header "Destination: http://nowhere" http://example.com
 
13. Web Login
 
 While not strictly just HTTP related, it still cause a lot of people problems
 so here's the executive run-down of how the vast majority of all login forms
 work and how to login to them using curl.
 
 It can also be noted that to do this properly in an automated fashion, you
 will most certainly need to script things and do multiple curl invokes etc.
 
 First, servers mostly use cookies to track the logged-in status of the
 client, so you will need to capture the cookies you receive in the
 responses. Then, many sites also set a special cookie on the login page (to
 make sure you got there through their login page) so you should make a habit
 of first getting the login-form page to capture the cookies set there.
 
 Some web-based login systems features various amounts of javascript, and
 sometimes they use such code to set or modify cookie contents. Possibly they
 do that to prevent programmed logins, like this manual describes how to...
 Anyway, if reading the code isn't enough to let you repeat the behavior
 manually, capturing the HTTP requests done by your browers and analyzing the
 sent cookies is usually a working method to work out how to shortcut the
 javascript need.
 
 In the actual <form> tag for the login, lots of sites fill-in random/session
 or otherwise secretly generated hidden tags and you may need to first capture
 the HTML code for the login form and extract all the hidden fields to be able
 to do a proper login POST. Remember that the contents need to be URL encoded
 when sent in a normal POST.
 
14. Debug
 
 Many times when you run curl on a site, you'll notice that the site doesn't
 seem to respond the same way to your curl requests as it does to your
 browser's.
 
 Then you need to start making your curl requests more similar to your
 browser's requests:
 
 * Use the --trace-ascii option to store fully detailed logs of the requests
   for easier analyzing and better understanding
 
 * Make sure you check for and use cookies when needed (both reading with
   --cookie and writing with --cookie-jar)
 
 * Set user-agent to one like a recent popular browser does
 
 * Set referer like it is set by the browser
 
 * If you use POST, make sure you send all the fields and in the same order as
   the browser does it. (See chapter 4.5 above)
 
 A very good helper to make sure you do this right, is the LiveHTTPHeader tool
 that lets you view all headers you send and receive with Mozilla/Firefox
 (even when using HTTPS).
 
 A more raw approach is to capture the HTTP traffic on the network with tools
 such as ethereal or tcpdump and check what headers that were sent and
 received by the browser. (HTTPS makes this technique inefficient.)
 
15. References
 
 RFC 2616 is a must to read if you want in-depth understanding of the HTTP
 protocol.
 
 RFC 3986 explains the URL syntax.
 
 RFC 2109 defines how cookies are supposed to work.
 
 RFC 1867 defines the HTTP post upload format.
 
 http://curl.haxx.se is the home of the cURL project


출처 : curl.haxx.se



Trackback 1 Comment 0
2010.02.10 18:27

The HTTP verb PUT under Apache: Safe or Dangerous?

"Is the HTTP verb PUT under Apache safe or dangerous?" This is a question I come across often, and have now run into it twice in the work on Atom. So is it safe? The answer is maybe.

Here are two such examples:

Using DELETE and PUT may be the "right thing to do" in an ideal world, but the fact of the matter is that a lot -- if not the vast majority -- of webservers do not allow these operations.

If anyone knows of a newer article describing HTTP PUT with apache, I would be very interested in seeing it. Because, due to my experience with PUT, you have to define a single PUTScript in httpd.conf, and if you PUT something to an apache server at the URI www.example.com/blog/entries/1 or something similar, apache passes all of the information to the PUTScript, not to anything else.

Both of the above quotes are from the Atom Wiki discussion of the use of PUT. A little digging reveals that the ApacheWeek article Publishing Pages with PUT is referenced most often when the danger of PUT is raised.

That ApacheWeek article does talk about the dangers of PUT and the cautions you need to follow when writing a script that does content publishing via PUT. That key part of that phrase is content publishing. That means that PUT is being used to upload arbitrary content to the server and the client is determining via the URI where the content should be stored. Now you can imagine how this might be dangerous, for example not correctly checking URI paths that include ../.. could let a malicious agent re-write your .bashrc.

Implementing a PUT script can be difficult and a security hazard in the context of content publishing, but that's the case because the client is choosing the target URI and the client could upload any content type. In the case of Web Services in general, and the AtomAPI in particular, PUT is used in a much narrower manner and avoids those potential security problems.

In the case of the AtomAPI PUT is only allowed on URIs that point to a pre-existing resource. The AtomAPI follows a general idiom for editing resources of doing a GET to retrieve the original XML, then a PUT on the same URI to upate that resource with the edited XML. No URIs are created by doing a PUT. PUT is not accepted on arbitrary URIs. This makes the use of PUT in the context of the AtomAPI just as safe as POST.

There are quite a few ways to configure Apache to process incoming requests. In particular it is possible to have a single script that handles all PUT requests below a chosen directory. This strategy, and all of the associated security concerns associated with it, are covered fully in the Publishing Pages with PUT.

When processing request with a CGI script all the PUT requests will come through. The verb is passed to the CGI program via the REQUEST_METHOD environment variable, and the program decides what to do with the content.

Using PUT propoerly has advantages in Web Service development. First, Apache lets you control security based on the verb using the Limit and LimitExcept directives, which let you restrict access controls based on the verb. Here is a sample of one of my .htaccess files that restricts the use of all verbs except GET to the CGI program Bulu.cgi.

<Files Bulu.cgi>
AuthType Basic
AuthName myrealm
AuthUserFile /path/to/my/password/file
  <LimitExcept GET>
  Require valid-user
  </LimitExcept>
</Files>

In addition, the Script directive can be used to dispatch to a CGI program based on the verb used:

Script PUT /cgi-bin/put.cgi

The second advantage using PUT brings is clarity. Given the idiom of using GET/PUT in tandem on a URI to edit resources PUT clearly signals what the interface is doing.

Resources

ApacheWeek: Publishing Pages with PUT

RestEchoApiPutAndDelete: Discussion on the use of PUT and DELETE in the AtomAPI.

mod_actions: An Apache module for controlling dispatching based on verb or content-type.

Configuring your WWW server to understand the PUT method, from the W3Cs Amaya project documentation.

WebDAV is also something you may be interested in if you are looking for ways to publish your content using HTTP. WebDAV stands for "Web-based Distributed Authoring and Versioning". It is a set of extensions to the HTTP protocol which allows users to collaboratively edit and manage files on remote web servers. Mod_dav in an Apache module that implements WebDAV.

출처 : http://bitworking.org


Trackback 19 Comment 0
2009.08.19 09:59

Secure File Upload Check List With PHP

Uploading file on your website is a very common thing nowadays. Image, zip and many other common file type are the usual things we want our users to be able to upload. However, potential evil files such as .exe, .php and other script files are those that we wish they can never be able to upload on to our server. And i am sure you are like me who will wonder whether my upload handler is secure enough to prevent attacks from coming in. In this article, i will try to list down most of the secure ways to protect our server and business from these potential threat. On the other hand, feel free to share your experience with the readers and me on the security tips you have.

Content Type Verification

Checking the content-type of a file is the first level of verification that many of us will do.

1.<?php
2.#easiest way to verify it is a image file
3.if(!eregi('image/', $_FILES['hpt_files']['type']))
4.{
5.     echo 'Please upload a valid file!';
6.     exit(0);
7.}
8.?>

Although, this can be easily bypass by attacker by changing the Content-type header which we will look at later. Nonetheless, it is something we must always check. Please take note that different MIME type may differ in different web browsers.

Verify Image File Content

Uploading image file is something most application will allow. An attacker can change the content-type to a valid one in order for your script to accept the file. Thus, we will have to ensure that this is really an image file by using getimagesize() in PHP.

1.<?php
2.$imageinfo = getimagesize($_FILES['uploadfile']['tmp_name']);
3.if($imageinfo['mime'] != 'image/gif' && $imageinfo['mime'] != 'image/jpeg' && isset($imageinfo))
4.{
5.     echo "Sorry, we only accept GIF and JPEG images\n";
6.     exit(0);
7.}
8.?>

You might want to check on other information as well. However, a file can be a proper GIF or JPEG image and at the same time a valid PHP script. Most image formats allow a text comment. It is possible to create a perfectly valid image file that contains some PHP code in the comment. How? By taking an image file (.jpg) and upload as a php extension file(.php). When getimagesize() look at it, it is a valid image file but when the PHP interpreter looks at it, the PHP code in the comment will be executed and other binary code will be discarded as junk (similar to HTML + PHP + JavaScript). Thus, getimagesize() only provides certain level of verification while many more have to be there in order to fully protect yourself.

Verify File Extension

This is something every upload handler in PHP must do. An attacker can fake the content-type of a file to the server, the extension must be a valid extension for PHP machine to interpret it correctly. Although this is not all of the security measure, this is definitely one of the important verification. I have included both white and black list on the code (although only one of them is required usually) since we won’t know what will happen to the server configuration especially in a shared hosting environment.

01.<?php
02.$filename = strtolower($_FILES['uploadfile']['name']);
03.$whitelist = array("jpg", "png", "gif", "jpeg"); #example of white list
04.$backlist = array("php", "php3", "php4", "phtml","exe"); #example of black list
05.if(!in_array(end(explode(".", $fileName)), $whitelist))
06.{
07.     echo "Invalid file type";
08.     exit(0);
09.}
10.if(in_array(end(explode(".", $fileName)), $backlist))
11.{
12.     echo "Invalid file type";
13.     exit(0);
14.}
15.?>

This way even if an attacker fake their way by changing the content-type, they will not be able to change the fact that the extension is required for the file to be interpreted by the machine. However, what file extensions will be passed on to the PHP interpreter will depend on the server configuration. A developer will have no knowledge or control over the web server configuration. Some web application may require that files with .gif or .jpg extension are interpreted by PHP. Thus, any comment in the image file will be interpreted by the PHP machine as a valid instruction to be executed.

Basically, we can’t guarantee that knowing what file extension is being interpreted by PHP machine can help eliminate all attack and it does not change at some point in the future, when some other application is installed on the web server.

The Upload Folder

We want to prevent users from requesting uploaded files directly. This means that the best place to keep these uploaded files is somewhere outside of the web root (www, public_html, etc.) or creating a directory under the web root and blocking web access to it in the Apache configuration or in a .htaccess file. If the attackers is able to upload some harmful file into your system, this will prevent them from executing the files and enter arbitrary code into the system as they are unable to access the location. Consider the following example,

01.<?php
02.$upload_dir = '/var/domainame/uploads/'; # Outside of web root
03.$upload_file = $uploaddir . basename($_FILES['uploadfile']['name']);
04.if (move_uploaded_file($_FILES['uploadfile']['tmp_name'], $uploadfile))
05.{
06.     echo "Upload Successfully.";
07.     exit(0);
08.}
09.else
10.{
11.     echo "Upload Fail";
12.     exit(0);
13.}
14.?>

This is somehow good but now the web server will not be able to access the directory too! Therefore, we need to provide another file for web server to access and display the file if necessary.

1.<?php
2.$upload_dir = '/var/domainame/uploads/'; # Outside of web root
3.$name = $_GET['name'];
4.readfile($uploaddir.$name);
5.?>

Now, both users and system will be able to access the directory provided that they know the name of the file. However, the above suffer from directory traversal vulnerability where a malicious user can use this script to read any readable file on the system. Consider the following example,

1.http://www.example.com/readfile.php?name=../secret/passwd

This will most probably return the password stored in the server. Therefore, always remember to secure your POST and GET in your PHP script.

IIS PUT Function

If you are running PHP on Microsoft IIS, you will have to take particular care on your writable web directories. Unlike Apache, Microsoft IIS supports ‘PUT’ HTTP requests, which allows users to upload files directly, without using an upload PHP page. However, ‘PUT’ requests can only be used to upload a file to your web directory if the file system permissions allow IIS to write to the directory and if IIS permission allowed writing for that directory.

To prevent this, we have to ensure that IIS permissions do not allow writing although we will have to allow the directory to be writable in order to upload using PHP script. This will caused one of the condition to fail and ‘PUT’ request will not be enable by IIS which is used to bypass all the check you have done on PHP script by using ‘PUT’ request to upload into your directory.

The Include Function

In some script, we tend to use the receive value from users to determine which file to include into the PHP script. This is usually not a good idea as the attacker can execute certain file in your web server. Consider the following example,

01.<?php
02.# ... some code here
03.if(isset($_COOKIE['lang']))
04.     $lang = $_COOKIE['lang'];
05.elseif (isset($_GET['lang']))
06.     $lang = $_GET['lang'];
07.elseif (isset($_GET['lang']))
08.     $lang = $_GET['lang'];
09.else
10.     $lang = 'english';
11.  
12.include("language/$lang.php");
13.# ... some more code here
14.?>

Assuming no filter is done on the data received, we determine the language and include the language file into the page which is a common piece of code for some of you. An attacker can take this flaws and enter a path on the URL to execute certain file in the system. Therefore, it is important to secure your upload function to prevent attacker from execute any file that are harmful to your system.(imagine they are able to upload certain shell or execution command and activate it via the URL)

Random File Name

We talk about how a file name should not be access directly by the users to prevent any form of attack. However, we can still access these file indirectly with the help of another script. But if the attacker do not know the name of the file that he have just uploaded, they might not be able to execute these arbitrary code into your web server. Thus, it is always good to randomly rename your file with md5 or other encryption algorithm. Consider the following example,

01.<?php
02.$filename = $_FILES[$uploadfile]["name"];
03.$save_path = '/var/domainame/uploads/'; # Outside of web root
04.$extension = end(explode(".", $filename)); #extension of the file
05.$renamed = md5($filename. time());      #rename of the file
06.if (!@move_uploaded_file($_FILES[$uploadfile]["tmp_name"], $save_path.$renamed. $extension))
07.{
08.     echo "File could not be saved.";
09.     exit(0);
10.}
11.?>

However, if the uploading is done by yourself through an upload function, renaming these uploaded files might not be good for SEO purposes. Thus, the security measure here are for upload function that allows visitors or external users to upload certain file into your web server. ( basically you don’t trust others than yourself )

Disable Script Execution

You can also try to disabled script execution on the uploaded folder where all the files go. You can do this by writing a .htacess file on the folder.

1.AddHandler cgi-script .php .php3 .php4 .phtml .pl .py .jsp .asp .htm .shtml .sh .cgi
2.Options -ExecCGI

This will gives you an extra layer of protection. You can also restrict certain file to be placed into the folder and only allows certain file to be placed into the folder. But remember that if some web application allows your ‘white list’ extension file to be interpreted by php machine, the chances of this protection might not be very useful. Nonetheless, this still serve as one of the many layer of protection for your web serverr.

HTML Upload Size

Although not all browsers do not support this but some still does. This can help provides certain level of protection against upload restriction.

1.<!-- allow 100kb -->
2.<input type="hidden" name="MAX_FILE_SIZE" value="100000" />

PHP Upload Size

We must also restrict the upload size on PHP to prevent any harmful file that is large enough to caused a sever damage to our server (any attack can caused a huge damage anyway). Checking the file size can also help you minimize the amount of disk space needed for your server.

01.<?php
02.#check for appropriate size with php.ini
03.$POST_MAX_SIZE = ini_get('post_max_size');
04.$mul = substr($POST_MAX_SIZE, -1);
05.$mul = ($mul == 'M' ? 1048576 : ($mul == 'K' ? 1024 : ($mul == 'G' ? 1073741824 : 1)));
06.if ($_SERVER['CONTENT_LENGTH'] > $mul*(int)$POST_MAX_SIZE && $POST_MAX_SIZE) $error = true;
07.$max_file_size_in_bytes = 2147483647;                  // 2GB in bytes
08.if(!$error)
09.{
10.     #restrict the limit
11.     $file_size = @filesize($_FILES[$upload_name]["tmp_name"]);
12.     if (!$file_size || $file_size > $max_file_size_in_bytes) {
13.          HandleError("File exceeds the maximum allowed size");
14.          exit(0);
15.     }
16.}
17.else
18.{
19.     HandleError("File exceeds the maximum allowed size in php.ini");
20.     exit(0);
21.}
22.?>

You can visit the PHP handing file uploads for more information.

Limit File Upload

DOS attack (Denial of service) might be one of the concern that you have. Users might be able to upload a lot of large files and consume all available disk space which prevented other users from using the service. Hence, certain restriction should be imposed to prevent such cases from happening. The application designer might want to implement a limit on
the size and number of files one user can upload in a given period (a day)

BLOB Type Storage

An alternative to storing files on the file system is keeping file data directly in the database as a BLOB. This approach has the advantage that everything related to the application is stored either under the web root or in the database. However, this approach probably wouldn’t be a good solution for large files or if the performance is critical.

Verify The Session

You may wish to impose certain security measure by having a session between the upload form and the upload handler to ensure that the user is authenticate to proceed with the upload.

Verify Upload

We must also verify that there is indeed a file being uploaded into the server to process the upload handler script.

01.<?php
02.if (!isset($_FILES[$upload_name])) {
03.     echo "No upload found in \$_FILES for " . $upload_name;
04.     exit(0);
05.} else if (isset($_FILES[$upload_name]["error"]) && $_FILES[$upload_name]["error"] != 0) {
06.     echo $uploadErrors[$_FILES[$upload_name]["error"]];
07.     exit(0);
08.} else if (!isset($_FILES[$upload_name]["tmp_name"]) || !@is_uploaded_file($_FILES[$upload_name]["tmp_name"])) {
09.     echo "Upload failed is_uploaded_file test.";
10.     exit(0);
11.} else if (!isset($_FILES[$upload_name]['name'])) {
12.     echo "File has no name.";
13.     exit(0);
14.}
15.?>

The above is an example to verify whether there is an upload file and whether it is secure to proceed the file that the user has uploaded.

Upload Folder within www

Don’t want your folder to be located outside of www or public_html? There is another solution for this. However, you might need to have dedicated or vps which has root access in order for this to work. Rather than giving write permission to the users, we give to apache instead. You can do this with a chown on the writable folder to apache or nobody and assign 770 permission.

Basically, this will disable public access to file in the directory. Short to say, external users will not be able to execute, read or write on the directory, only Apache is allowed to since it is the owner of the folder.

My Upload Handler

This is the upload handler that i usually rely on which you might be interested.

01.<?php
02.     #check for session
03.     if (isset($_POST["PHPSESSID"]))
04.          session_id($_POST["PHPSESSID"]);
05.     else if (isset($_GET["PHPSESSID"]))
06.          session_id($_GET["PHPSESSID"]);
07.     else
08.     {
09.          HandleError("No Session was found.");
10.     }
11.     session_start();
12.// Check post_max_size (http://us3.php.net/manual/en/features.file-upload.php#73762)
13.     $POST_MAX_SIZE = ini_get('post_max_size');
14.     $unit = strtoupper(substr($POST_MAX_SIZE, -1));
15.     $multiplier = ($unit == 'M' ? 1048576 : ($unit == 'K' ? 1024 : ($unit == 'G' ? 1073741824 : 1)));
16.  
17.     if ((int)$_SERVER['CONTENT_LENGTH'] > $multiplier*(int)$POST_MAX_SIZE && $POST_MAX_SIZE)
18.          HandleError("POST exceeded maximum allowed size.");
19.  
20.// Settings
21.     $save_path = getcwd() . "/uploads/";                   // The path were we will save the file (getcwd() may not be reliable and should be tested in your environment)
22.     $upload_name = "Filedata";                                  // change this accordingly
23.     $max_file_size_in_bytes = 2147483647;                  // 2GB in bytes
24.     $whitelist = array("jpg", "png", "gif", "jpeg");  // Allowed file extensions
25.     $backlist = array("php", "php3", "php4", "phtml","exe"); // Restrict file extensions
26.     $valid_chars_regex = 'A-Za-z0-9_-\s ';// Characters allowed in the file name (in a Regular Expression format)
27.  
28.// Other variables
29.     $MAX_FILENAME_LENGTH = 260;
30.     $file_name = "";
31.     $file_extension = "";
32.     $uploadErrors = array(
33.        0=>"There is no error, the file uploaded with success",
34.        1=>"The uploaded file exceeds the upload_max_filesize directive in php.ini",
35.        2=>"The uploaded file exceeds the MAX_FILE_SIZE directive that was specified in the HTML form",
36.        3=>"The uploaded file was only partially uploaded",
37.        4=>"No file was uploaded",
38.        6=>"Missing a temporary folder"
39.     );
40.  
41.// Validate the upload
42.     if (!isset($_FILES[$upload_name]))
43.          HandleError("No upload found in \$_FILES for " . $upload_name);
44.     else if (isset($_FILES[$upload_name]["error"]) && $_FILES[$upload_name]["error"] != 0)
45.          HandleError($uploadErrors[$_FILES[$upload_name]["error"]]);
46.     else if (!isset($_FILES[$upload_name]["tmp_name"]) || !@is_uploaded_file($_FILES[$upload_name]["tmp_name"]))
47.          HandleError("Upload failed is_uploaded_file test.");
48.     else if (!isset($_FILES[$upload_name]['name']))
49.          HandleError("File has no name.");
50.  
51.// Validate the file size (Warning: the largest files supported by this code is 2GB)
52.     $file_size = @filesize($_FILES[$upload_name]["tmp_name"]);
53.     if (!$file_size || $file_size > $max_file_size_in_bytes)
54.          HandleError("File exceeds the maximum allowed size");
55.  
56.     if ($file_size <= 0)
57.          HandleError("File size outside allowed lower bound");
58.// Validate its a MIME Images (Take note that not all MIME is the same across different browser, especially when its zip file)
59.     if(!eregi('image/', $_FILES[$upload_name]['type']))
60.          HandleError('Please upload a valid file!');
61.  
62.// Validate that it is an image
63.     $imageinfo = getimagesize($_FILES[$upload_name]['tmp_name']);
64.     if($imageinfo['mime'] != 'image/gif' && $imageinfo['mime'] != 'image/jpeg' && $imageinfo['mime'] != 'image/png' && isset($imageinfo))
65.          HandleError("Sorry, we only accept GIF and JPEG images");
66.  
67.// Validate file name (for our purposes we'll just remove invalid characters)
68.     $file_name = preg_replace('/[^'.$valid_chars_regex.']|\.+$/i', "", strtolower(basename($_FILES[$upload_name]['name'])));
69.     if (strlen($file_name) == 0 || strlen($file_name) > $MAX_FILENAME_LENGTH)
70.          HandleError("Invalid file name");
71.  
72.// Validate that we won't over-write an existing file
73.     if (file_exists($save_path . $file_name))
74.          HandleError("File with this name already exists");
75.  
76.// Validate file extension
77.     if(!in_array(end(explode(".", $file_name)), $whitelist))
78.          HandleError("Invalid file extension");
79.     if(in_array(end(explode(".", $file_name)), $backlist))
80.          HandleError("Invalid file extension");
81.// Rename the file to be saved
82.     $file_name = md5($file_name. time());
83.  
84.// Verify! Upload the file
85.     if (!@move_uploaded_file($_FILES[$upload_name]["tmp_name"], $save_path.$file_name)) {
86.          HandleError("File could not be saved.");
87.     }
88.     exit(0);
89.  
90./* Handles the error output. */
91.function HandleError($message) {
92.     echo $message;
93.     exit(0);
94.}
95.?>

Conclusion

This is not a full proof solution for your file upload handler. However, this can used as references and also discussion that can help enhance the overall security of our web application today.


원문 : http://hungred.com/2009/08/17/useful-information/secure-file-upload-check-list-php/


Trackback 4 Comment 0