I wonder if Zeppelin is a good solution with these steps:
- Get the data directly via sql (already works) but the integrated display possibilities are nice but not sufficient for me.
- A user can adjust several filters (e.g. column1 between 1 and 3 and so) using the AngularJS view (https://zeppelin.apache.org/docs/0.7.3/displaysystem/front-end-angular.html).
- Read the filter ranges from the last paragraph and do data wrangling with pythonâs pandas with some filters and save it a new dataframe.
- Use this filtered dataframe and plot the data with a local plotly.js file (I have the pure plotly JS code). In general, it could work with an %angular paragraph as Using d3.js with Apache Zeppelin (even without the code from âelseâ, so d3 seem to be useable within zeppelin out of the box). But I wonder
- how to pass the dataframe from Python to the Angular node. According to https://zeppelin.apache.org/docs/0.7.3/displaysystem/back-end-angular.html I have to use Spark to use the variables; so it seems that I have to integrate a spark wrapping paragraph. After that do I have to write code that converts the dataframe to JSON or similar?
What do you think or do you have other/better ideas?