Eclipse dataflow plugin errors when running the WordCount example pipeline

[rafaelsf80]

Hi,
I'm trying to run the wordcount example pipeline with the dataflow plugin (1.2.0) for eclipse Neon.2 (4.6.2), and got this error when trying to Run a new configuration:

An internal error occurred during: "Update Hierarchy".
Tried to create a TypeHierarchyPipelineOptionsHierarchy for a Java Project my-artifact-rafa where no PipelineOptions type exists.

Updating the artifact google-cloud-dataflow-java-sdk-all from 1.9.0 to 2.0.0-beta1 or 2.0.0-beta2 on pom.xml shows this error instead when running the pipeline:

Exception in thread "main" java.lang.Error: Unresolved compilation problems: 
	PipelineOptionsFactory cannot be resolved
	Pipeline cannot be resolved to a type
	Pipeline cannot be resolved
	TextIO cannot be resolved
	ParDo cannot be resolved
	TextIO cannot be resolved

	at com.google.cloud.dataflow.examples.WordCount.main(WordCount.java:191)

Any guidance would be appreciated.

Thanks,
Rafa

[davorbonaci]

@tgroh, can you please take a look?

[tgroh]

Usually, this occurs due to the maven view of the dependencies being out of sync with the declared dependencies of the project. This means the PipelineOptions type is not present within the classpath of the Eclipse Project. The maven dependencies are likely to not have been properly resolved.

From the project context menu, select the Maven -> Update Project item, check the "Force Update of Snapshots/Releases" checkbox, and run the update. This should ensure the Dataflow Jar is on the classpath, which will enable the Dataflow Eclipse Plugin to resolve the base PipelineOptions class, and the remainder of the PipelineOptions class hierarchy.

Updating to a 2.0.0-beta-x version will not work, as the namespaces for all of the underlying PTransforms and Pipeline methods has changed from com.google.cloud.dataflow to org.apache.beam

[rafaelsf80]

Hi, Thanks for the answer. The issue was the one you described: the maven view was out of sync. However the Force Update did not solve the issue itself. Since I only use maven for this particular project (for nothing else), I removed the $HOME/.m2 content and then after a mvn clean install, everything works properly now.
Thanks for the suggestion,
Rafa

[davorbonaci]

Thanks for confirming, @rafaelsf80!