File upload with Ajax and Cross-domain

I’ve recently tried to upload a file from a client to a server and faced the problem of cross-domain. Indeed, uploading is done with ajax, to be able to analyze the response. So, I had to solve 2 problems: uploading with ajax and cross-domain.

Concerning cross-domain, I’ve found many different solutions but none of them really fit my need since I needed to add informations in the header. (I think there were other reasons, but cannot remember). I find it easier to make a two step upload. First, I upload the file to my simple PushState server, then the server upload the file to the other website. This server is built using NodeJs, express and request. Upload is made with request and its form parameter which allows us to do multipart/form-data.
Here is the server:

var express = require('express');
var request = require('request');
var fs = require('fs');
var app = express();

app.use(express.logger());
app.use(express.bodyParser({uploadDir:'./tmp'}));
app.use(app.router);
app.use(express.static('./public'));
app.use(function(req, res) {
  fs.createReadStream('./public/index.html').pipe(res);
});

app.post('/upload', function(req, res, next) {
  //Configure the request
  var opts = {
    url: "http://your/url",
    method: "POST",
    headers: {
      login: "yourLogin",
      password: "yourPassword"
    }
  };

  var x = request(opts);
  var form = x.form();

  //Optionnal: Add a name and a type to form
  var fileName = req.files.file.name.split('.');
  form.append('name', fileName[0]);
  form.append('type', fileName[fileName.length - 1]);

  //Add file to the request
  form.append('file', fs.createReadStream(req.files.file.path));

  //Pipe response from website to the client
  x.pipe(res);
});

app.listen(4242, function() {
  console.log('Server running!');
});

Uploading with ajax is done thank to jquery.iframe-transport (just add it in your code).
Concerning ajax, code is really simple:

$.ajax({
  url: "http://localhost:4242/upload",
  type: 'POST',
  files: $('.fileUpload'),
  iframe: true,
  success: function(data, textStatus, jqXHR) {
    //Do whatever you want with the response
  },
  error: function(err) {
    console.error(err);
    console.log("Error while uploading the document. :/");
  }
});

The html behind:

<form enctype="multipart/form-data">
  <input class='fileUpload' type="file" name="file"/>
</form>

In this case, I listen for the event change input.fileUpload which indicates that a file has been chosen and then, call a function executing the ajax part. It would also be possible to react on click on a button.

Other solutions concerning cross-domain:                                                      http://stackoverflow.com/questions/3076414/ways-to-circumvent-the-same-origin-policy                                                                                  https://github.com/blueimp/jQuery-File-Upload/wiki/Cross-domain-uploads

NPM: noproxy Configuration

For security reasons, every firms are generally using a proxy to manage their internet connection. Coding behind a proxy is fine. Most softwares are able to deal with it. You just have to set it in the environment variable of your GNU/Linux OS or directly in the software.

But there is one problem. Because every request must go through the proxy, it’s impossible to request url based in the internal network. There is only one solution: disable proxy, make request, enable proxy. If you are using npm until now, that’s what you have to do.

In order to make this process easier, I’ve made a pull request which add a noproxy configuration. When npm fetches a package from a given url, the hostname is compared with noproxy configuration. If there is a match, the request is made without proxy, if not proxy is used.

The noproxy configuration looks for a variable named « noproxy » in npm configuration.
« noproxy » is a string containing hostnames. So, this list of hostnames will not ever go through a proxy.
Code concerned:

var proxy = null
if(npm.config.get("noproxy").search(remote.hostname) === -1) {
  if (remote.protocol !== "https:" || 
      !(proxy = npm.config.get("https-proxy"))) {
    proxy = npm.config.get("proxy")
  }
}

var opts = { url: remote
          , proxy: proxy
          , strictSSL: npm.config.get("strict-ssl")
          , ca: remote.host === regHost ? 
                                npm.config.get("ca") : undefined
          , headers: { "user-agent": npm.config.get("user-agent") }}

var req = request(opts)

At the moment, I’m still waiting for Isaacs to add the pull request in npm.
I hope it will be added soon.

Update:                                                                                                                                     Isaacs has pointed out that it’s good for the npm bit but some modifications are still needed in npm-registry-client and npmconf . So here are new additions:

  • npmconf:
 , "noproxy" : process.env.NO_PROXY || process.env.no_proxy ||  "null"
 , "no-proxy" : ["null", String]
  • npm-registry-client:
  var p = this.conf.get('proxy')
  var sp = this.conf.get('https-proxy') || p
  var np = this.conf.get('noproxy')

  if(np.search(remote.hostname) === -1) {
    opts.proxy = remote.protocol === "https:" ? sp : p
  }

I hope everything is now in order so that noproxy configuration can be added to npm :).

[NodeJs] Spawn ou la création de processus fils

Un petit mot sur la façon de créer des processus fils sous node.
Pas compliqué et bien pratique, pour réaliser le cas de test d’une nouvelle fonctionnalité de npm par exemple.

Regardons le prototype:
child_process.spawn(command, [args], [options])
command correspond au programme appelé, ls, grep ou autre…
[args] est un tableau contenant tous les arguments qui seront passés au programme appelé via command.
[options] Plusieurs possibilités, dont le fait de pouvoir définir des règles pour le comportement de stdin, stdout et stderr.

Exemple:

//Set the no-proxy configuration
var noProxy = spawn(node, [npm, 'config', 'set', 'noproxy=localhost']
, { stdio: 'ignore' });

Ici, on crée un processus fils node avec pour paramètres: npm config noproxy=localhost.
On indique également que les entrées et sorties seront ignorées: { stdio: ‘ignore’ }.

Il est bien sûr intéressant de pouvoir effectuer une action particulière dès que notre fils se termine. Ceci est possible grâce à .on(‘exit’, function() { //CODE } ).
Soit avec l’exemple précédent:

noProxy.on('exit', function (code) {
console.log('noProxy process exited with exit code '+code);
});

Concernant les entrées/sorties, il est possible de laisser le processus écrire dans celles du processus parent en spécifiant cette fois: { stdio: ‘inherit’ }.
Un autre point utile consiste à récupérer le contenu de la sortie standard du processus fils.
Pour ce faire, on utilise les options suivantes: [‘ignore’, ‘stream’, ‘ignore’].
stdin et stderr sont ignorés, seule reste stdout.
On peut donc récupérer les données via .stdout.on(‘data’, function (data) { //CODE } ) , où data contient les données.
Pour reprendre l’exemple de npm, on peut ainsi récupérer la valeur de la variable de configuration proxy et la stocker dans une variable locale proxySave

var getProxy = spawn(node, [npm, 'config', 'get', 'proxy']
, ['ignore', 'stream', 'ignore']);
getProxy.stdout.on('data', function (data) {
proxySave = data.toString().trim();
});

Voilà pour les principales infos concernant spawn. Pour en savoir plus, direction la doc officielle ;).