pathwaycom/pathway

[WEBSITE] Bug in 'Pathway in Minutes: Quick ETL Examples'

Closed this issue · 3 comments

raphaelreimann commented

What is your question or problem? Please describe.
I was going through the User Guide on pathway.com and noticed that the source code for the 'First Realtime App with Pathway' has a bug.

alerts_table = joined_values.filter(  # joined_values is not defined
    pw.this.value > pw.this.threshold
).select(pw.this.name, pw.this.value)

Describe what you would like to happen
Between the table join and the filter step you might want to define the joined values from the joined table.

Cheers!

dxtrous commented
izulin commented

True - thanks a lot for the catch! Did fixing the typo (changing joined_values.filter to joined_table.filter) resolve the problem for you? We will publish a documentation fix directly.

There seems to be a typo in skipping the join().select() steps.

One solution is to write

# Joining tables on the column name
joined_table = measurements_table.join( # The left table is measurements_table (pw.left)
    thresholds_table,                   # The right table is thresholds_table (pw.right)
    pw.left.name==pw.right.name,        # The join is done on the column name of each table 
).select(
    pw.left.*,                          # All the columns of measurements are kept
    pw.right.threshold                  # The threshold column of the threshold table is kept
)

# Filtering value strictly higher than the threshold.
alerts_table = joined_table.filter(
    pw.this.value > pw.this.threshold
).select(pw.this.name, pw.this.value) # Only name and value fields are kept

another is to use the filter directly on join_result and write

# Joining tables on the column name
join_result = measurements_table.join( # The left table is measurements_table (pw.left)
    thresholds_table,                   # The right table is thresholds_table (pw.right)
    pw.left.name==pw.right.name,        # The join is done on the column name of each table 
)

# Filtering value strictly higher than the threshold.
alerts_table = join_result.filter(
    pw.this.value > pw.this.threshold
).select(pw.this.name, pw.this.value) # Only name and value fields are kept
olruas commented

This has been fixed and should be online by tomorrow.
Thank you @raphaelreimann.

  • 👍1