Parse etymologies from Wiktionary

[skalyan91]

Write a function that does the following:

1. Take a language and a wordform (you may need to map from ISO code to language name; CLiCS might have a mapping table, otherwise use the one from Glottolog).
2. Visit the Wiktionary page for the wordform and go to the section for the appropriate language.
3. Go to the "Etymology" subsection (or if there are multiple etymology subsections, loop over them).
4. Extract the first sentence, of the form "From <language name> <ancestral form>".
5. Take the <language name> <ancestral form> pairing, and use those as input for another function call. (I.e. use tail recursion.)
6. Keep chasing etymologies until you’ve hit a dead end. Along the way, save each etymological link to a table (with the fields "Language", "Wordform", "Source language", and "Source wordform").

Once we have a function that does the above, we can just loop it over all the words in NorthEuraLex.

[Tavalam]

Here's a note
You say: fill the table with the fields "Language", "Wordform", "Source language", and "Source wordform".
Yet some Wiktionary etymology paragraphs include also info on semantics of source form
e.g.

maison

From Middle French, from Old French maisun, meson, inherited from Latin mānsiō, mānsiōnem (“abode, home, dwelling”), from maneō (“remain, stay”) (whence also French manoir).

In this partic. example I guess it can be ignored, because redundant with the linked entry; but in some cases the quoted meaning may help disambiguate the target sense:
e.g.

palm (plant)

specifies a certain meaning of Latin palma

From Middle English palme, from Old English palm, palma (“palm-tree, palm-branch”), from Latin palma (“palm-tree, palm-branch, palm of the hand”), from Proto-Indo-European *pl̥h₂meh₂, *plām- (“palm of the hand”). Cognate with Dutch palm, German Palme, Danish palme, Icelandic pálmur (“palm”).
while

palm (hand)

words the semantics differently:

From Middle English palme, paume, from Old French palme, paulme, paume (“palm of the hand, ball, tennis”), from Latin palma (“palm of the hand, hand-breadth”), from Proto-Indo-European *palam-, *plām- (“palm of the hand”). Cognate with Ancient Greek παλάμη (palámē, “palm of the hand”), Old English folm (“palm of the hand”), Old Irish lám (“hand”).

Should these be ignored altogether by the script? or incorporated in some way? (even if the reference gloss for a word is a separate column)

[skalyan91]

@Tavalam This depends on how we wish to treat homophony. I think the simplest and most easily defensible approach is to not treat homophony as different from polysemy. What this would mean in the case of English palm is the following:

1. English palm colexifies the “plant” and “surface of hand” senses.
2. English palm comes from both Middle English palme and Middle English paume.
3. Middle English palme has the same colexification as Modern English palm, whereas Middle English paume only has the “hand” sense.
4. Middle English palme comes from both Old English palm, palma and Old French palme, paulme, paume. (In other words, it has five etymologies!)
5. Middle English paume comes from Old French palme, paulme, paume.
6. The Old English words only have the “plant” sense, and the Old French words only have the “hand” sense.
7. Both the Old English and Old French words go back to Latin palma, which once again colexifies the “plant” and “hand” senses—but this is colexification is not a direct ancestor of the colexification in Middle English onwards! The link was broken along the way.

I think this is all the information that we need; it is unnecessary to encode the information that Middle English palme (plant) comes from Old English, whereas Middle English palme (hand) comes from Old French, since the Old English etymon only has the “plant” sense, and the Old French etymon only has the “hand” sense. In other words, we can infer the fact that the Middle English word is homonymous and not polysemous, without having to manually encode this assumption anywhere.

I hope this makes some kind of sense.