6 Commits

Author SHA1 Message Date
Thomas Leister
45c8da4111 Merge pull request #2 from xamanu/master
Avoid serving overview of main directory
2018-07-08 11:28:14 +02:00
Felix Delattre
9054a5db4c Avoid serving overview of main directory 2018-07-07 02:49:12 +02:00
Thomas Leister
819d418fb1 Adds sets proxy buffer to off 2018-07-04 15:50:23 +02:00
Thomas Leister
8c3743a450 Adds hint about Ejabberd 2018-07-04 08:07:51 +02:00
Thomas Leister
6470a3579b Fix mistake in cronjob command 2018-07-02 14:15:32 +02:00
Thomas Leister
e50279a564 Updates README: Adds statement of Matthew Wild 2018-07-02 13:01:03 +02:00
3 changed files with 43 additions and 3 deletions

View File

@@ -2,9 +2,21 @@
A simple file server for handling XMPP http_upload requests. This server is meat to be used with the Prosody [mod_http_upload_external](https://modules.prosody.im/mod_http_upload_external.html) module. A simple file server for handling XMPP http_upload requests. This server is meat to be used with the Prosody [mod_http_upload_external](https://modules.prosody.im/mod_http_upload_external.html) module.
**Why should I use this server?** *(This module can also be used with future versions of Ejabberd: https://github.com/processone/ejabberd/commit/fface33d54f24c777dbec96fda6bd00e665327fe)*
* Prosody's integrated http_upload server seems to be memory leaking. ## Why should I use this server?
* Prosody developers recommend using http_upload_external instead of http_upload (Matthew Wild on the question if http_upload is memory leaking):
> "BTW, I am not aware of any memory leaks in the HTTP upload code. However it is known to be very inefficient.
> That's why it has a very low upload limit, and **we encourage people to use mod_http_upload_external instead**.
> We set out to write a good XMPP server, not HTTP server (of which many good ones already exist), so our HTTP server is optimised for small bits of data, like BOSH and websocket.
> Handling large uploads and downloads was not a goal (and implementing a great HTTP server is not a high priority for the project compared to other things).
> **Our HTTP code buffers the entire upload into memory.
> More, it does it in an inefficient way that can use up to 4x the actual size of the data (if the data is large).
> So uploading a 10MB file can in theory use 40MB RAM.**
> But it's not a leak, the RAM is later cleared and reused. [...]
> The GC will free the memory at some point, but the OS may still report that Prosody is using that memory due to the way the libc allocator works.
> Most long lived processes behave this way (only increasing RAM, rarely decreasing)."
* This server works without any script interpreters or additional dependencies. It is delivered as a binary. * This server works without any script interpreters or additional dependencies. It is delivered as a binary.
* Go is very good at serving HTTP requests. * Go is very good at serving HTTP requests.
@@ -140,6 +152,7 @@ Create a new config file ```/etc/nginx/sites-available/uploads.myserver.tld```:
location /upload/ { location /upload/ {
proxy_pass http://127.0.0.1:5050/upload/; proxy_pass http://127.0.0.1:5050/upload/;
proxy_request_buffering off;
} }
} }
@@ -161,7 +174,7 @@ Reload Nginx:
Prosody Filer has no immediate knowlegde over all the stored files and the time they were uploaded, since no database exists for that. Also Prosody is not capable to do auto deletion if *mod_http_upload_external* is used. Therefore the suggested way of purging the uploads directory is to execute a purge command via a cron job: Prosody Filer has no immediate knowlegde over all the stored files and the time they were uploaded, since no database exists for that. Also Prosody is not capable to do auto deletion if *mod_http_upload_external* is used. Therefore the suggested way of purging the uploads directory is to execute a purge command via a cron job:
@daily find /var/lib/prosody/uploads -maxdepth 0 -type d -mtime +28 | xargs rm -rf @daily find /home/prosody-filer/uploads -maxdepth 0 -type d -mtime +28 | xargs rm -rf
This will delete uploads older than 28 days. This will delete uploads older than 28 days.

View File

@@ -122,6 +122,10 @@ func handleRequest(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", contentType) w.Header().Set("Content-Type", contentType)
} else if r.Method == "GET" { } else if r.Method == "GET" {
contentType := mime.TypeByExtension(filepath.Ext(fileStorePath)) contentType := mime.TypeByExtension(filepath.Ext(fileStorePath))
if fileStorePath == "" {
http.Error(w, "403 Forbidden", 403)
return
}
if contentType == "" { if contentType == "" {
contentType = "application/octet-stream" contentType = "application/octet-stream"
} }

View File

@@ -183,3 +183,26 @@ func TestDownloadGet(t *testing.T) {
t.Errorf("handler returned wrong status code: got %v want %v. HTTP body: %s", status, http.StatusOK, rr.Body.String()) t.Errorf("handler returned wrong status code: got %v want %v. HTTP body: %s", status, http.StatusOK, rr.Body.String())
} }
} }
func TestEmptyGet(t *testing.T) {
// Set config
readConfig("config.toml", &conf)
// Create request
req, err := http.NewRequest("GET", "", nil)
if err != nil {
t.Fatal(err)
}
rr := httptest.NewRecorder()
handler := http.HandlerFunc(handleRequest)
// Send request and record response
handler.ServeHTTP(rr, req)
// Check status code
if status := rr.Code; status != http.StatusForbidden {
t.Errorf("handler returned wrong status code: got %v want %v. HTTP body: %s", status, http.StatusForbidden, rr.Body.String())
}
}