phrase/phrase-cli

Pulling YML files brings outdated values

bernawil opened this issue · 7 comments

bernawil commented

I am seeing key values coming with outdated values after pulling yml files after a push.

To test this I created a test project

this is my .phraseapp

phraseapp:
  access_token: my-token
  project_id: 1afcb9912f3e68c4777275e78cd18512
  push:
    sources:
      - file: ./config/locales/en.yml
        params:
          locale_id: en
          file_format: yml
          update_translations: true
          update_descriptions: true
  pull:
    targets:
      - file: ./config/locales/<locale_name>.yml
        params:
          file_format: yml

this is my en.yml

en:
  first_key:
    nest_2:
      empty: text 3
    value:
      one: one - THIS HAS BEEN UPDATED
  fourth_key:
    nest: some text 2
    nest_2:
      empty: text 3
  second_key:
    value:
      one: two
  third_key: some text 2

There, I update first_key.value.one

en:
  first_key:
    nest_2:
      empty: text 3
    value:
      one: one - THIS HAS BEEN UPDATED

Then push the changes:

phrase push
Uploading config/locales/en.yml... done!
Check upload Id: dd76d44b6f841236f238d4f21f901179, filename: en.yml for information about processing results: https://app.phrase.com/accounts/imkey-bv/projects/bernardo-test/uploads/dd76d44b6f841236f238d4f21f901179

Now, pull those changes and en.yml comes with the value outdated. To double, check I download the file directly.

phrase locales download --id en --project_id 1afcb9912f3e68c4777275e78cd18512 --file_format yml

---
en:
  first_key:
    nest_2:
      empty: text 3
    value:
      one: one
  fourth_key:
    nest: some text 2
    nest_2:
      empty: text 3
  second_key:
    value:
      one: two
  third_key: some text 2

Result: first_key.value.one still outdated

Now, try downloading simple_json.

phrase locales download --id en --project_id 1afcb9912f3e68c4777275e78cd18512 --file_format simple_json

{
  "first_key.nest_2.empty": "text 3",
  "first_key.value": {
    "one": "one - THIS HAS BEEN UPDATED"
  },
  "first_key.value.one": "one",
  "fourth_key.nest": "some text 2",
  "fourth_key.nest_2.empty": "text 3",
  "second_key.value": {
    "one": "two"
  },
  "second_key.value.one": "two",
  "third_key": "some text 2"
}

first_key.value.one is now up to date!

This mirrors what happens in the Web UI: the editor shows updated values and downloading simple json format too. But downloading yml brings these outdated values.

This looks something wrong in the conversion between the data representation and yml format instead of the data persistence itself. For the record, I see similar outdated values in other formats like json (simple json is correct).

theSoenke commented

Hi @bernawil, uploads are processed in the background and it can take from a few seconds to a few minutes until the content is updated the project. What you could try is running phrase push --wait. This will make the command wait until the uploads have been processed

bernawil commented
jablan commented

Could it be a pluralization problem? AFAIK, JSON exports show plurals as leaves (.one, .zero, .many...) in the document tree, and it's indistinguishable from an actual key which contains .one in its name. So, a pluralized key named foo and a non-pluralized key named foo.one would result with clashing JSON values.

@bernawil make sure you don't combine pluralized keys with keys named .one etc.

bernardo-talentinc commented

Could it be a pluralization problem? AFAIK, JSON exports show plurals as leaves (.one, .zero, .many...) in the document tree, and it's indistinguishable from an actual key which contains .one in its name. So, a pluralized key named foo and a non-pluralized key named foo.one would result with clashing JSON values.

@bernawil make sure you don't combine pluralized keys with keys named .one etc.

I doubt it, I created that example to reproduce the issue, but I've seen it happen with keys named differently too.
Anyway, if that was the case then the API should validate against such naming.

jablan commented

The naming itself isn't wrong. It just doesn't go well when combined with pluralization and nested export/import.

  "first_key.value": {
    "one": "one - THIS HAS BEEN UPDATED"
  },
  "first_key.value.one": "one",

There's the problem, you have two keys which result with the same part in the nested export/import file.

bernawil commented

The naming itself isn't wrong. It just doesn't go well when combined with pluralization and nested export/import.

  "first_key.value": {
    "one": "one - THIS HAS BEEN UPDATED"
  },
  "first_key.value.one": "one",

There's the problem, you have two keys which result with the same part in the nested export/import file.

Again, I'm seeing the same with other keys, this was just a test to catch it.
But then, this is obviously a bug in the API, it shouldn't let you push something that causes this, or at least throw a warning.

bernardo-talentinc commented

The naming itself isn't wrong. It just doesn't go well when combined with pluralization and nested export/import.

  "first_key.value": {
    "one": "one - THIS HAS BEEN UPDATED"
  },
  "first_key.value.one": "one",

There's the problem, you have two keys which result with the same part in the nested export/import file.

@jablan just noticed this:
this is the original en.yml I push:

en:
  first_key:
    nest_2:
      empty: text 3
    value:
      one: one - THIS HAS BEEN UPDATED

there's only one first_key.value.one. The simple json shows both a "first_key.value.one" and an object leaf with the same. One updated, the other not.

  "first_key.value": {
    "one": "one - THIS HAS BEEN UPDATED"
  },
  "first_key.value.one": "one",

There's obviously something wrong the yml parsing that makes duplicated keys.