Trying to walk a folder with a recursive promise. How can I discard recursive returns, keeping only the last?

Hi, people :smiley:
First of all, I’m new here, so sorry if I make some terrible mistake :þ (I try hard to don’t do it).

Well, I will go to the point. This is a problem I’m having with NodeJS (I guess it fails in this category, right?)

I’m trying to write a little script to go over a folder (& subfolders) and get a list of all the files on it with certain ext name (.html). The purpose really is apply some transformation to those files (so really I won’t need to get the list, I guess, but, for know, I’m trying to get it. The code works and get the list of all files, but my problem is that I guess the results repeated some times.

I’m trying (because I’m pretty new in it, and in everything, I guess) to use promises, via Bluebird and I think that the problem of the repeating comes from this line

return stats.isDirectory() ? walk(fileName, ext, callback): (extname === ext)

where I do a recursive call to the walk function, to walk a subfolder when one is found. But I don’t understand really why this happen, I think it should happen.

What I do in this function is, basically, read the actual dir, filter (Bluebird method), returns a Promise) the list of files, keeping those that have the wished ext (could be a file… but also a folder) and those without extension (folder candidates) and then checking if they’re folders and, if so, digging into them with walk to repeat this process and get their files. Then, with each (also a Bluebird method, returns a Promise) I add those files to an array, so that way I get the wanted list.

Thing is, as I said previously, that I’m dont getting a list, but a series of lists. I asumme every list is the result of every call to walk

Here, the code:

const Promise = require('bluebird')
      fs = require('fs')
      path = require('path')
const argv = require('yargs')
.default('dir', './public')
.alias('d', 'dir')
.argv

const dir = argv.dir
      ext = '.html'

Promise.promisifyAll(fs)

let files = []

function walk (dir, ext, callback) {
  fs.readdirAsync(dir).filter(function(fileName) {
    let extname = path.extname(fileName)
    fileName = dir + '/' + fileName
    if ([ext, ''].indexOf(extname) > -1) {
      return fs.statAsync(fileName)
        .then(function(stats) {
          // If fileName is a dir, look into it, else print file with given ext
          return stats.isDirectory() ? walk(fileName, ext, callback): (extname === ext)
        }).catch(console.log)
    } else {
      return false
    }
  }).each(fileName => files.push(dir + '/' + fileName))
  .then(results => console.log(files))
}

walk(dir, ext, console.log)

Below I add an example package.json for running it locally and a silly shell script to create a simple folder tree to try it. With a file tree example, the script should be executed with node index.js -d folder_to_walk. This was going to be a Gist, easy to clone and all that, but Discourse keeps insisting that I’m forbidden to post more than two links form being a new user… even after I delete one link after other, so… I’m sorry.

Other files

package.json

{
  "name": "tiquitaca",
  "version": "0.0.1",
  "description": "Walking",
  "main": "index.js",
  "license": "ISC",
  "dependencies": {
    "bluebird": "^3.4.6",
    "yargs": "^6.0.0"
  }
}

tree.sh

#!/bin/bash
folders=('prueba/este/script/illo' 'prueba/a/ver' 'prueba/luke/' 'prueba/usa/la/fuerza')
files=('prueba/index.html' 'prueba/este/main.css' 'prueba/luke/vader.html' 'prueba/usa/leia.html' 'prueba/usa/la/fuerza/yoda.html')

for folder in "${folders[@]}"; do
  mkdir -p "$folder"
done
for file in "${files[@]}"; do
  touch "$file"
done

Question is: how can I discard the rest of results and keep only the last ?

I though that, maybe, I could turn walkitself into a promise, but this doesn’t work:

/* at the end */
...
 }).each(fileName => files.push(`${dir}/${fileName}`))
  .then(results => files)  // just return the value, not console.log it, do it later
}

Promise.promisify(walk)
walk(dir, ext, console.log).then(files => console.log(files))

Probably the solution is easy, I just don’t see it. Thanks for your help and time in advance ñ_ñ.

Note: I’m aware of packages like klaw (well, I was aware after I started trying to write this xD), but well, still wanna try this.

1 Like

Well, I don’t need anymore an answer about this, because this was to solve a problem with an Static Site Generator that I won’t be using right now, so I comment it, for prevent you to waste your time ;). I can change the title neither close the thread, so…

In any case, it would be nice to know how you would resolve it.

2 Likes

This topic was automatically closed 91 days after the last reply. New replies are no longer allowed.