Structured Data Across Wikimedia/Image Suggestions/Data Pipeline
 | This page in a nutshell: The image suggestions data pipeline is a wide effort that spans several teams: we describe here the Structured Data side of the road. |
General Architecture
Who Does What
| Team | Pipeline components |
|---|---|
| Data Engineering | Data Lake |
| Data Platform | pipeline infrastructure, Cassandra |
| Platform Operations and Dumps | Image Suggestions API |
| Search Platform | Search indices update |
| Structured Data | Airflow job |
The Structured Data Side
TODO