pyspark.sql.functions.to_timestamp#

pyspark.sql.functions.to_timestamp(col, format=None)[source]#

Converts a Column into pyspark.sql.types.TimestampType using the optionally specified format. Specify formats according to datetime pattern. By default, it follows casting rules to pyspark.sql.types.TimestampType if the format is omitted. Equivalent to col.cast("timestamp").

Added in version 2.2.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters:

colColumn or column name: column values to convert. A column that evaluates to a string, date, timestamp, or numeric.
format: literal string, optional: format to use to convert timestamp values. A column that evaluates to a string.

Returns:

Column: timestamp value as pyspark.sql.types.TimestampType type. Returns a column that evaluates to a timestamp.

See also

pyspark.sql.functions.to_date()
pyspark.sql.functions.to_timestamp_ltz()
pyspark.sql.functions.to_timestamp_ntz()
pyspark.sql.functions.to_utc_timestamp()
pyspark.sql.functions.to_unix_timestamp()
pyspark.sql.functions.try_to_timestamp()
pyspark.sql.functions.date_format()

Examples

Example 1: Convert string to a timestamp

>>> import pyspark.sql.functions as sf
>>> df = spark.createDataFrame([('1997-02-28 10:30:00',)], ['t'])
>>> df.select(sf.to_timestamp(df.t)).show()
+-------------------+
|    to_timestamp(t)|
+-------------------+
|1997-02-28 10:30:00|
+-------------------+

Example 2: Convert string to a timestamp with a format

>>> import pyspark.sql.functions as sf
>>> df = spark.createDataFrame([('1997-02-28 10:30:00',)], ['t'])
>>> df.select(sf.to_timestamp(df.t, 'yyyy-MM-dd HH:mm:ss')).show()
+------------------------------------+
|to_timestamp(t, yyyy-MM-dd HH:mm:ss)|
+------------------------------------+
|                 1997-02-28 10:30:00|
+------------------------------------+