Create pages from all files in a directory

[jan10]

Thanks for the create plugin!
Is there a way to create pages from all files in _data folders?

For example:

_data/books/
books.yml
Books-2.yml

_data/shops/
-shop.yml
-xyz.yml

[avillafiorita]

you just need to populate the page_gen configuration option with one record per file.

For instance:

page_gen:
  - data: books/books.yml
    template: <<name of the template to use to generate the page for book.yml>>
    name: <<field used to generate the filename>>
    dir: <<directory in which files are to be generated>>
    extension: <<extension used to generate the filename>>
    filter: <<property to filter data records by>>
  - data: books/Books-2.yml
    template: <<name of the template to use to generate the page for book.yml>>
    name: <<field used to generate the filename>>
    dir: <<directory in which files are to be generated>>
    extension: <<extension used to generate the filename>>
    filter: <<property to filter data records by>>
  ...

if you want all pages to land in the same directory, you can try using the same value for dir in the different entries (I have not tested it, though)

For instance:

page_gen:
  - data: books/books.yml
    template: <<name of the template to use to generate the page for book.yml>>
    name: <<field used to generate the filename>>
    dir: books/
    extension: <<extension used to generate the filename>>
    filter: <<property to filter data records by>>
  - data: books/Books-2.yml
    template: <<name of the template to use to generate the page for book.yml>>
    name: <<field used to generate the filename>>
    dir: books/
    extension: <<extension used to generate the filename>>
    filter: <<property to filter data records by>>
  ...

Let me know if it solves your issue!

[jan10]

Thanks for your feedback! I explained it in a misleading way, I'm sorry.
I do not know the total number or names of the files in the folder.

Real world example:
I use the Jekyll-Contentful-Data-Import to import all the data i create in the contentful backend.
Now I want to create a page from each .yaml.

1clkacMDbu6IcOeUES0Ky2.yaml:

---
sys:
  id: 1clkacMDbu6IcOeUES0Ky2
  created_at: !ruby/object:DateTime 2018-10-07 16:28:14.617000000 Z
  updated_at: !ruby/object:DateTime 2018-10-07 16:28:14.617000000 Z
  content_type_id: systems
  revision: 1
name: z-wave

The result that I would like to have:
_site/systems/z-wave/index.html

[avillafiorita]

Hi, now I see!
This is something which the plugin can't do (the specification in _config.yml requires to list all files in advance).

One possibility is writing a script which looks in the _data folder and outputs the configuration for datapage_gen. You would then need to add the configuration thus generated to your _config.yml file.

The script could be something along the lines of:

yml_files = Dir.glob("_data/**.yml")
puts "page_gen:"
yml_files.each do |file|
  puts <<EOS
  - data: #{file.gsub("_data", "").gsub(".yml", "")}
    name: SET ME BY HAND
EOS
end

I did not try the code nor the output it generates, so please, double check!

The solution makes sense if your data files are not generated at run-time and if they don't change too often, which might be the case for a once-off migration from a CMS.

Another possibility would be that of extending datapage_gen to manage wildcards. This requires to decide also some default values for fields usually specified by hand, like, e.g., the name field.

[chinanderm]

+1 the ability to have wildcards and then be able to reference that wildcard value still in the configuration. Example:

  - data: 'schedules.*year*.games'
    template: 'game'
    name: 'week'
    dir: 'schedule/*year*/'

In this case, *year* would refer to the filename.

[avillafiorita]

it sounds like an interesting feature to add. Close to Jekyll collections, but not quite the same.

I need a suggestion with the syntax, however... if we want to be able to output in different directories, we should use regexps, which already support pattern matching and substitution. The example you make becomes:

- data: schedules\.(.*)year(.*)\.games
    template: 'game'
    name: 'week'
    dir: 'schedule/\1year\2/'

Another possibility is using glob-like pattern matching for data and omitting the destination dir, which could be, by default, the matching string or the same for all matched data. The syntax is simpler and less expressive than the previous case:

- data: schedules.*year*.games
    template: 'game'
    name: 'week'

Thoughts?

[chinanderm]

@avillafiorita Thanks for the quick reply. I vote for the regexp method since it's more expressive (which I think is a good thing in this very specific use case).

Btw, didn't say it before, thanks for the plugin!

[jhvanderschee]

You guys are looking for individual_entry_files: false in the jekyll-contentful-data-import plugin. This will prevent all those files and solves this problem/issue.