Databricks Certified Associate Developer for Apache Spark

In Apache Spark, which process is responsible for coordinating the execution of jobs and scheduling tasks on executors?

• AExecutor
• BDriver
• CWorker
• DCluster Manager

Which of the following is a narrow transformation in Spark?

• AgroupBy
• Bjoin (sort-merge)
• Cfilter
• Ddistinct

Which Spark action triggers the execution of transformations and returns data to the driver?

• Adf.select()
• Bdf.filter()
• Cdf.collect()
• Ddf.withColumn()

What does lazy evaluation mean in Spark?

• ATransformations are not executed until an action is called
• BTransformations run sequentially without optimization
• CData is not loaded from disk
• DOnly small DataFrames are evaluated

What does this PySpark code do? df.select('name', 'age').filter('age > 18').show()

• ASelects name and age columns, filters rows where age > 18, displays top rows
• BUpdates name and age columns
• CDeletes rows where age > 18
• DReturns an error because filter uses a string

Which PySpark code renames a column named 'FirstName' to 'first_name'?

• Adf.withColumnRenamed('FirstName', 'first_name')
• Bdf.rename('FirstName', 'first_name')
• Cdf.alias('FirstName', 'first_name')
• Ddf.withColumn('FirstName', F.col('first_name'))

Which method adds a new column to a DataFrame based on a computed expression?

• Adf.addColumn('newcol', expr)
• Bdf.withColumn('newcol', F.col('a') + F.col('b'))
• Cdf.new('newcol', expr)
• Ddf.insertColumn('newcol', expr)

Given df has columns id, name, age, which code returns only the id and age columns cast to Long and Integer respectively?

• Adf.select(F.col('id').cast('long'), F.col('age').cast('int'))
• Bdf.alias('id', 'long').alias('age', 'int')
• Cdf.cast('id', 'long').cast('age', 'int')
• Ddf.select(F.cast('id','long'), F.cast('age','int'))

Which method removes duplicate rows across all columns in a DataFrame?

• Adf.distinct()
• Bdf.unique()
• Cdf.removeDuplicates()
• Ddf.dedup()

Which F.* function replaces null values with a specified default?

• AF.coalesce
• BF.lit
• CF.ifnull
• DBoth F.coalesce and F.ifnull are valid for this

Which PySpark code filters rows where name is not null?

• Adf.filter(df.name != None)
• Bdf.filter(df.name.isNotNull())
• Cdf.filter(df.name != '')
• Ddf.filter(df.name is not None)

Which action returns the first n rows of a DataFrame as a list of Row objects on the driver?

• Adf.take(n)
• Bdf.show(n)
• Cdf.head()
• Ddf.limit(n)

Which Spark function converts a string column to uppercase?

• AF.upper(col)
• BF.toUpper(col)
• CF.uppercase(col)
• Dcol.up()

Given df.groupBy('dept').agg(F.avg('salary').alias('avg_salary')), what does this compute?

• AThe average salary per department
• BThe total salary across all employees
• CThe number of employees per department
• DThe maximum salary across all departments

Which PySpark expression counts distinct values of a column?

• AF.count('col')
• BF.countDistinct('col')
• CF.sum('col')
• DF.avg('col')

Which approximate aggregation uses HyperLogLog for fast distinct counts on huge datasets?

• AF.approx_count_distinct
• BF.countDistinct
• CF.sum
• DF.percentile_approx

Which join type returns only rows with matching keys in both DataFrames?

• Ainner
• Bleft_outer
• Cfull_outer
• Dleft_anti

Which join returns rows from the left DataFrame that have no match in the right?

• Aleft_semi
• Bleft_anti
• Cfull_outer
• Dcross

Which hint forces Spark to broadcast a small DataFrame to all executors, avoiding shuffle during a join?

• AF.broadcast(small_df)
• Bsmall_df.cache()
• Csmall_df.repartition(1)
• Dspark.sql.shuffle.partitions=1

Which Spark component optimizes logical query plans into physical execution plans?

• ACatalyst
• BTungsten
• CSpark SQL
• DHive Metastore

Which setting enables Adaptive Query Execution (AQE) in Spark?

• Aspark.sql.adaptive.enabled=true
• Bspark.catalyst.adaptive.enabled=true
• Cspark.aqe.enabled=true
• Dspark.conf.set('adaptive', true)

Which method reads a Parquet file into a DataFrame?

• Aspark.read.parquet('path')
• Bspark.read.format('parquet').load('path')
• CBoth A and B are valid
• Dspark.parquet.read('path')

Which DataFrame write mode replaces existing data at the target path?

• Aappend
• Boverwrite
• Cignore
• Derrorifexists

Which code writes a DataFrame partitioned by 'year' and 'month'?

• Adf.write.partitionBy('year', 'month').parquet(path)
• Bdf.write.bucketBy(10, 'year', 'month').parquet(path)
• Cdf.partitionBy('year', 'month').write.parquet(path)
• Ddf.write.parquet(path, partitions=['year','month'])

Which is the correct way to define a custom schema for spark.read.csv?

• Aschema = StructType([StructField('name', StringType(), True), StructField('age', IntegerT…
• Bschema = {'name': 'string', 'age': 'int'}
• Cschema = [('name', 'string'), ('age', 'int')]
• Dschema = 'name string, age int'

Why is explicit schema preferred over schema inference on large CSV files?

• AIt reduces the scan because Spark doesn't need a preliminary pass to infer types
• BInference is always wrong
• CInference does not support CSV
• DExplicit schema enables write but not read

Which option instructs spark.read.csv to treat the first line as headers?

• Aheader=True
• Bfirst_row=True
• Ccolumns='header'
• Dnames='header'

Which Window specification computes a 7-day rolling average?

• AWindow.partitionBy('user_id').orderBy('date').rowsBetween(-6, 0)
• BWindow.orderBy('date')
• CWindow.partitionBy('date')
• DWindow.over('user_id')

Which window function assigns a unique sequential number within each partition?

• AF.row_number()
• BF.rank()
• CF.dense_rank()
• DF.ntile(10)

Which window function returns the value of a column from the previous row within a partition?

• AF.lag(col)
• BF.lead(col)
• CF.first(col)
• DF.previous(col)

Which PySpark code creates a temporary view for Spark SQL queries?

• Adf.createOrReplaceTempView('my_view')
• Bdf.registerTempTable('my_view')
• Cspark.catalog.createView('my_view', df)
• DBoth A and B (B is deprecated but works)

Which runs an SQL query on a registered view?

• Aspark.sql('SELECT * FROM my_view')
• Bspark.query('SELECT * FROM my_view')
• Csql.run('SELECT * FROM my_view')
• Dspark.dataframe('SELECT * FROM my_view')

Which is the correct difference between repartition and coalesce?

• Arepartition can increase or decrease partition count via a full shuffle; coalesce can onl…
• BBoth do the same thing
• Ccoalesce always triggers a full shuffle while repartition does not
• Drepartition works only on RDDs

What does df.cache() do?

• APersists the DataFrame with default MEMORY_AND_DISK storage level for reuse
• BForces eager execution of all transformations
• CWrites the DataFrame to a cache folder on disk
• DEquivalent to collect()

Which StorageLevel serializes data in memory (less memory, more CPU to deserialize)?

• AMEMORY_ONLY
• BMEMORY_ONLY_SER
• CDISK_ONLY
• DMEMORY_AND_DISK

What is the purpose of df.unpersist()?

• AFrees memory/disk used by a cached DataFrame
• BDeletes the DataFrame from disk permanently
• CConverts the DataFrame to RDD
• DTriggers evaluation

Which UDF approach has the BEST performance in PySpark?

• APython UDF with @udf
• Bpandas_udf (vectorized, Arrow-based)
• CUDF written in SQL as a string
• DPlain Python lambda in map