Zaloni Ideas
Status Will not implement
Created by Pragyan Choudhury
Created on Aug 2, 2021

Ability to run PySpark scripts from hdfs location

Currently, if pyspark code is present on hdfs, we need to first bring it to a shared directory on the local file system and then run pyspark action on it. However it would be very useful if Arena was able to run scripts directly from hdfs instead of copying and then running it.

Support Ticket PS-32255
Customer Impact Minor inconvenience
  • Attach files