However, there's a small problem with our dataset: most of the string columns have either leading or trailing white spaces. Here's how you can correct this:

import pyspark.sql.functions as func

for col, typ in census.dtypes:
    if typ == 'string':
        census = census.withColumn(
            col
            , func.ltrim(func.rtrim(census[col]))
        )

We loop through all the columns in the census DataFrame.

If the type of the column is equal to string, we apply two functions: .ltrim(...), which removes any leading whitespaces in a string, and .rtrim(...), which removes any trailing whitespaces. The .withColumn(...) method does not append any new columns as we reuse the same name for the column: col.