Advertisement
Mad_Axell

Untitled

Feb 27th, 2023
44
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.40 KB | None | 0 0
  1. messages = spark.read.parquet("/user/madaxell/data/events/events.parquet")\
  2. .where("event_type = 'reaction'")\
  3. .select(F.col("event.message_id").alias("message_id"),
  4. F.col("event.reaction_from").alias("user_id"),
  5. #F.explode(F.col("event.tags")).alias("tag"),
  6. F.col("event.tags").alias("tags"),
  7. F.col("event.reaction_type").alias("reaction"))\
  8. .where("tags IS NOT NULL")
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement