Higher-order functions allow us to pass functions as arguments or return them as results, which can significantly enhance the modularity of a machine learning pipeline. For instance, we can create a generic function that applies various preprocessing steps on data sets, allowing for easy adjustments and testing of different approaches without altering the core pipeline structure.