Array, map and struct functions
Every Spark SQL array, map, struct, lambda and vector function on Databasin Sail: whether it is supported, how it differs from Apache Spark, and examples with results.
This page lists every Spark SQL array, map, struct, lambda and vector function: 60 work on Databasin Sail as in Apache Spark and 1 works with a difference noted below. Each function shows its status on Sail, its usage, and examples whose results on Sail match Spark's.
Every supported function also works from PySpark through pyspark.sql.functions when PySpark has a wrapper for it. The Sail SQL reference lists every function page.
Array
array · array_append · array_compact · array_contains · array_distinct · array_except · array_insert · array_intersect · array_join · array_max · array_min · array_position · array_prepend · array_remove · array_repeat · array_size · array_union · arrays_overlap · arrays_zip · flatten · get · sequence · shuffle · slice · sort_array
array
Supported.
array(expr, ...): Returns an array with the given elements.
Since Spark 1.1.0.
SELECT array(1, 2, 3);
-- [1,2,3]
array_append
Supported. Results match Spark; the auto-generated result column name can differ in some examples.
array_append(array, element): Add the element at the end of the array passed as first argument. Type of element should be similar to type of the elements of the array. Null element is also appended into the array. But if the array passed, is NULL output is NULL
Since Spark 3.4.0.
SELECT array_append(array('b', 'd', 'c', 'a'), 'd');
-- ["b","d","c","a","d"]
SELECT array_append(array(1, 2, 3, null), null);
-- [1,2,3,null,null]
SELECT array_append(CAST(null as Array<Int>), 2);
-- NULL
array_compact
Supported.
array_compact(array): Removes null values from the array.
Since Spark 3.4.0.
SELECT array_compact(array(1, 2, 3, null));
-- [1,2,3]
SELECT array_compact(array("a", "b", "c"));
-- ["a","b","c"]
array_contains
Supported.
array_contains(array, value): Returns true if the array contains the value.
Since Spark 1.5.0.
SELECT array_contains(array(1, 2, 3), 2);
-- true
array_distinct
Supported.
array_distinct(array): Removes duplicate values from the array.
Since Spark 2.4.0.
SELECT array_distinct(array(1, 2, 3, null, 3));
-- [1,2,3,null]
array_except
Supported.
array_except(array1, array2): Returns an array of the elements in array1 but not in array2, without duplicates.
Since Spark 2.4.0.
SELECT array_except(array(1, 2, 3), array(1, 3, 5));
-- [2]
array_insert
Supported.
array_insert(x, pos, val): Places val into index pos of array x. Array indices start at 1. The maximum negative index is -1 for which the function inserts new element after the current last element. Index above array size appends the array, or prepends the array if index is negative, with 'null' elements.
Since Spark 3.4.0.
SELECT array_insert(array(1, 2, 3, 4), 5, 5);
-- [1,2,3,4,5]
SELECT array_insert(array(5, 4, 3, 2), -1, 1);
-- [5,4,3,2,1]
SELECT array_insert(array(5, 3, 2, 1), -4, 4);
-- [5,4,3,2,1]
array_intersect
Supported.
array_intersect(array1, array2): Returns an array of the elements in the intersection of array1 and array2, without duplicates.
Since Spark 2.4.0.
SELECT array_intersect(array(1, 2, 3), array(1, 3, 5));
-- [1,3]
array_join
Supported.
array_join(array, delimiter[, nullReplacement]): Concatenates the elements of the given array using the delimiter and an optional string to replace nulls. If no value is set for nullReplacement, any null value is filtered.
Since Spark 2.4.0.
SELECT array_join(array('hello', 'world'), ' ');
-- hello world
SELECT array_join(array('hello', null ,'world'), ' ');
-- hello world
SELECT array_join(array('hello', null ,'world'), ' ', ',');
-- hello , world
array_max
Supported.
array_max(array): Returns the maximum value in the array. NaN is greater than any non-NaN elements for double/float type. NULL elements are skipped.
Since Spark 2.4.0.
SELECT array_max(array(1, 20, null, 3));
-- 20
array_min
Supported.
array_min(array): Returns the minimum value in the array. NaN is greater than any non-NaN elements for double/float type. NULL elements are skipped.
Since Spark 2.4.0.
SELECT array_min(array(1, 20, null, 3));
-- 1
array_position
Supported.
array_position(array, element): Returns the (1-based) index of the first matching element of the array as long, or 0 if no match is found.
Since Spark 2.4.0.
SELECT array_position(array(312, 773, 708, 708), 708);
-- 3
SELECT array_position(array(312, 773, 708, 708), 414);
-- 0
array_prepend
Supported. Results match Spark; the auto-generated result column name can differ in some examples.
array_prepend(array, element): Add the element at the beginning of the array passed as first argument. Type of element should be the same as the type of the elements of the array. Null element is also prepended to the array. But if the array passed is NULL output is NULL
Since Spark 3.5.0.
SELECT array_prepend(array('b', 'd', 'c', 'a'), 'd');
-- ["d","b","d","c","a"]
SELECT array_prepend(array(1, 2, 3, null), null);
-- [null,1,2,3,null]
SELECT array_prepend(CAST(null as Array<Int>), 2);
-- NULL
array_remove
Supported.
array_remove(array, element): Remove all elements that equal to element from array.
Since Spark 2.4.0.
SELECT array_remove(array(1, 2, 3, null, 3), 3);
-- [1,2,null]
array_repeat
Supported.
array_repeat(element, count): Returns the array containing element count times.
Since Spark 2.4.0.
SELECT array_repeat('123', 2);
-- ["123","123"]
array_size
Supported.
array_size(expr): Returns the size of an array. The function returns null for null input.
Since Spark 3.3.0.
SELECT array_size(array('b', 'd', 'c', 'a'));
-- 4
array_union
Supported.
array_union(array1, array2): Returns an array of the elements in the union of array1 and array2, without duplicates.
Since Spark 2.4.0.
SELECT array_union(array(1, 2, 3), array(1, 3, 5));
-- [1,2,3,5]
arrays_overlap
Supported.
arrays_overlap(a1, a2): Returns true if a1 contains at least a non-null element present also in a2. If the arrays have no common element and they are both non-empty and either of them contains a null element null is returned, false otherwise.
Since Spark 2.4.0.
SELECT arrays_overlap(array(1, 2, 3), array(3, 4, 5));
-- true
arrays_zip
Supported.
arrays_zip(a1, a2, ...): Returns a merged array of structs in which the N-th struct contains all N-th values of input arrays.
Since Spark 2.4.0.
SELECT arrays_zip(array(1, 2, 3), array(2, 3, 4));
-- [{"0":1,"1":2},{"0":2,"1":3},{"0":3,"1":4}]
SELECT arrays_zip(array(1, 2), array(2, 3), array(3, 4));
-- [{"0":1,"1":2,"2":3},{"0":2,"1":3,"2":4}]
flatten
Supported.
flatten(arrayOfArrays): Transforms an array of arrays into a single array.
Since Spark 2.4.0.
SELECT flatten(array(array(1, 2), array(3, 4)));
-- [1,2,3,4]
get
Supported.
get(array, index): Returns element of array at given (0-based) index. If the index points outside of the array boundaries, then this function returns NULL.
Since Spark 3.4.0.
SELECT get(array(1, 2, 3), 0);
-- 1
SELECT get(array(1, 2, 3), 3);
-- NULL
SELECT get(array(1, 2, 3), -1);
-- NULL
sequence
Supported. Results match Spark; the auto-generated result column name can differ in some examples.
sequence(start, stop, step): Generates an array of elements from start to stop (inclusive), incrementing by step. The type of the returned elements is the same as the type of argument expressions.
Supported types are: byte, short, integer, long, date, timestamp.
The start and stop expressions must resolve to the same type. If start and stop expressions resolve to the 'date' or 'timestamp' type then the step expression must resolve to the 'interval' or 'year-month interval' or 'day-time interval' type, otherwise to the same type as the start and stop expressions.
start: an expression. The start of the range.stop: an expression. The end the range (inclusive).step: an optional expression. The step of the range. By default step is 1 if start is less than or equal to stop, otherwise -1. For the temporal sequences it's 1 day and -1 day respectively. If start is greater than stop then the step must be negative, and vice versa.
Since Spark 2.4.0.
SELECT sequence(1, 5);
-- [1,2,3,4,5]
SELECT sequence(5, 1);
-- [5,4,3,2,1]
SELECT sequence(to_date('2018-01-01'), to_date('2018-03-01'), interval 1 month);
-- [2018-01-01,2018-02-01,2018-03-01]
SELECT sequence(to_date('2018-01-01'), to_date('2018-03-01'), interval '0-1' year to month);
-- [2018-01-01,2018-02-01,2018-03-01]
shuffle
Supported.
shuffle(array): Returns a random permutation of the given array.
Note from Spark: The function is non-deterministic.
Since Spark 2.4.0.
SELECT shuffle(array(1, 20, 3, 5));
-- for example: [3,1,5,20]
SELECT shuffle(array(1, 20, null, 3));
-- for example: [20,null,3,1]
slice
Supported.
slice(x, start, length): Subsets array x starting from index start (array indices start at 1, or starting from the end if start is negative) with the specified length.
Since Spark 2.4.0.
SELECT slice(array(1, 2, 3, 4), 2, 2);
-- [2,3]
SELECT slice(array(1, 2, 3, 4), -2, 2);
-- [3,4]
sort_array
Supported.
sort_array(array[, ascendingOrder]): Sorts the input array in ascending or descending order according to the natural ordering of the array elements. NaN is greater than any non-NaN elements for double/float type. Null elements will be placed at the beginning of the returned array in ascending order or at the end of the returned array in descending order.
Since Spark 1.5.0.
SELECT sort_array(array('b', 'd', null, 'c', 'a'), true);
-- [null,"a","b","c","d"]
SELECT sort_array(array('b', 'd', null, 'c', 'a'), false);
-- ["d","c","b","a",null]
Collection
cardinality · concat · element_at · reverse · size · try_element_at
cardinality
Supported.
cardinality(expr): Returns the size of an array or a map. This function returns -1 for null input only if spark.sql.ansi.enabled is false and spark.sql.legacy.sizeOfNull is true. Otherwise, it returns null for null input. With the default settings, the function returns null for null input.
Since Spark 2.4.0.
SELECT cardinality(array('b', 'd', 'c', 'a'));
-- 4
SELECT cardinality(map('a', 1, 'b', 2));
-- 2
concat
Supported.
concat(col1, col2, ..., colN): Returns the concatenation of col1, col2, ..., colN.
Note from Spark: Concat logic for arrays is available since 2.4.0.
Since Spark 1.5.0.
SELECT concat('Spark', 'SQL');
-- SparkSQL
SELECT concat(array(1, 2, 3), array(4, 5), array(6));
-- [1,2,3,4,5,6]
element_at
Supported.
element_at(array, index): Returns element of array at given (1-based) index. If Index is 0, Spark will throw an error. If index < 0, accesses elements from the last to the first. The function returns NULL if the index exceeds the length of the array and spark.sql.ansi.enabled is set to false. If spark.sql.ansi.enabled is set to true, it throws ArrayIndexOutOfBoundsException for invalid indices.
element_at(map, key): Returns value for given key. The function returns NULL if the key is not contained in the map.
Since Spark 2.4.0.
SELECT element_at(array(1, 2, 3), 2);
-- 2
SELECT element_at(map(1, 'a', 2, 'b'), 2);
-- b
reverse
Supported, differs from Spark. reverse on BINARY input fails (Spark 4.2 doc example hex(reverse(x'CAFE'))); strings and arrays match Spark.
reverse(expr): Returns a reversed string, a binary value with bytes in reverse order, or an array with reverse order of elements.
Note from Spark: Reverse logic for arrays is available since 2.4.0. Reverse logic for binary is available since 4.2.0.
Since Spark 1.5.0.
SELECT reverse('Spark SQL');
-- LQS krapS
SELECT reverse(array(2, 1, 4, 3));
-- [3,4,1,2]
size
Supported.
size(expr): Returns the size of an array or a map. This function returns -1 for null input only if spark.sql.ansi.enabled is false and spark.sql.legacy.sizeOfNull is true. Otherwise, it returns null for null input. With the default settings, the function returns null for null input.
Since Spark 1.5.0.
SELECT size(array('b', 'd', 'c', 'a'));
-- 4
SELECT size(map('a', 1, 'b', 2));
-- 2
try_element_at
Supported.
try_element_at(array, index): Returns element of array at given (1-based) index. If Index is 0, Spark will throw an error. If index < 0, accesses elements from the last to the first. The function always returns NULL if the index exceeds the length of the array.
try_element_at(map, key): Returns value for given key. The function always returns NULL if the key is not contained in the map.
Since Spark 3.3.0.
SELECT try_element_at(array(1, 2, 3), 2);
-- 2
SELECT try_element_at(map(1, 'a', 2, 'b'), 2);
-- b
Map
map · map_concat · map_contains_key · map_entries · map_from_arrays · map_from_entries · map_keys · map_values · str_to_map
map
Supported.
map(key0, value0, key1, value1, ...): Creates a map with the given key/value pairs.
Since Spark 2.0.0.
SELECT map(1.0, '2', 3.0, '4');
-- {1.0:"2",3.0:"4"}
map_concat
Supported.
map_concat(map, ...): Returns the union of all the given maps
Since Spark 2.4.0.
SELECT map_concat(map(1, 'a', 2, 'b'), map(3, 'c'));
-- {1:"a",2:"b",3:"c"}
map_contains_key
Supported.
map_contains_key(map, key): Returns true if the map contains the key.
Since Spark 3.3.0.
SELECT map_contains_key(map(1, 'a', 2, 'b'), 1);
-- true
SELECT map_contains_key(map(1, 'a', 2, 'b'), 3);
-- false
map_entries
Supported.
map_entries(map): Returns an unordered array of all entries in the given map.
Since Spark 3.0.0.
SELECT map_entries(map(1, 'a', 2, 'b'));
-- [{"key":1,"value":"a"},{"key":2,"value":"b"}]
map_from_arrays
Supported.
map_from_arrays(keys, values): Creates a map with a pair of the given key/value arrays. All elements in keys should not be null
Since Spark 2.4.0.
SELECT map_from_arrays(array(1.0, 3.0), array('2', '4'));
-- {1.0:"2",3.0:"4"}
map_from_entries
Supported.
map_from_entries(arrayOfEntries): Returns a map created from the given array of entries.
Since Spark 2.4.0.
SELECT map_from_entries(array(struct(1, 'a'), struct(2, 'b')));
-- {1:"a",2:"b"}
map_keys
Supported.
map_keys(map): Returns an unordered array containing the keys of the map.
Since Spark 2.0.0.
SELECT map_keys(map(1, 'a', 2, 'b'));
-- [1,2]
map_values
Supported.
map_values(map): Returns an unordered array containing the values of the map.
Since Spark 2.0.0.
SELECT map_values(map(1, 'a', 2, 'b'));
-- ["a","b"]
str_to_map
Supported.
str_to_map(text[, pairDelim[, keyValueDelim]]): Creates a map after splitting the text into key/value pairs using delimiters. Default delimiters are ',' for pairDelim and ':' for keyValueDelim. Both pairDelim and keyValueDelim are treated as regular expressions.
Since Spark 2.0.1.
SELECT str_to_map('a:1,b:2,c:3', ',', ':');
-- {"a":"1","b":"2","c":"3"}
SELECT str_to_map('a');
-- {"a":null}
Struct
named_struct
Supported.
named_struct(name1, val1, name2, val2, ...): Creates a struct with the given field names and values.
Since Spark 1.5.0.
SELECT named_struct("a", 1, "b", 2, "c", 3);
-- {"a":1,"b":2,"c":3}
struct
Supported.
struct(col1, col2, col3, ...): Creates a struct with the given field values.
Since Spark 1.4.0.
SELECT struct(1, 2, 3);
-- {"col1":1,"col2":2,"col3":3}
Lambda (higher-order)
aggregate · array_sort · exists · filter · forall · map_filter · map_zip_with · reduce · transform · transform_keys · transform_values · zip_with
aggregate
Supported.
aggregate(expr, start, merge, finish): Applies a binary operator to an initial state and all elements in the array, and reduces this to a single state. The final state is converted into the final result by applying a finish function.
Since Spark 2.4.0.
SELECT aggregate(array(1, 2, 3), 0, (acc, x) -> acc + x);
-- 6
SELECT aggregate(array(1, 2, 3), 0, (acc, x) -> acc + x, acc -> acc * 10);
-- 60
array_sort
Supported. Results match Spark; the auto-generated result column name can differ in some examples.
array_sort(expr, func): Sorts the input array. If func is omitted, sort in ascending order. The elements of the input array must be orderable. NaN is greater than any non-NaN elements for double/float type. Null elements will be placed at the end of the returned array. Since 3.0.0 this function also sorts and returns the array based on the given comparator function. The comparator will take two arguments representing two elements of the array. It returns a negative integer, 0, or a positive integer as the first element is less than, equal to, or greater than the second element. If the comparator function returns null, the function will fail and raise an error.
Since Spark 2.4.0.
SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then -1 when left > right then 1 else 0 end);
-- [1,5,6]
SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is null and right is null then 0 when left is null then -1 when right is null then 1 when left < right then 1 when left > right then -1 else 0 end);
-- ["dc","bc","ab"]
SELECT array_sort(array('b', 'd', null, 'c', 'a'));
-- ["a","b","c","d",null]
exists
Supported.
exists(expr, pred): Tests whether a predicate holds for one or more elements in the array.
Since Spark 2.4.0.
SELECT exists(array(1, 2, 3), x -> x % 2 == 0);
-- true
SELECT exists(array(1, 2, 3), x -> x % 2 == 10);
-- false
SELECT exists(array(1, null, 3), x -> x % 2 == 0);
-- NULL
SELECT exists(array(0, null, 2, 3, null), x -> x IS NULL);
-- true
SELECT exists(array(1, 2, 3), x -> x IS NULL);
-- false
filter
Supported.
filter(expr, func): Filters the input array using the given predicate.
Note from Spark: The inner function may use the index argument since 3.0.0.
Since Spark 2.4.0.
SELECT filter(array(1, 2, 3), x -> x % 2 == 1);
-- [1,3]
SELECT filter(array(0, 2, 3), (x, i) -> x > i);
-- [2,3]
SELECT filter(array(0, null, 2, 3, null), x -> x IS NOT NULL);
-- [0,2,3]
forall
Supported.
forall(expr, pred): Tests whether a predicate holds for all elements in the array.
Since Spark 3.0.0.
SELECT forall(array(1, 2, 3), x -> x % 2 == 0);
-- false
SELECT forall(array(2, 4, 8), x -> x % 2 == 0);
-- true
SELECT forall(array(1, null, 3), x -> x % 2 == 0);
-- false
SELECT forall(array(2, null, 8), x -> x % 2 == 0);
-- NULL
map_filter
Supported.
map_filter(expr, func): Filters entries in a map using the function.
Since Spark 3.0.0.
SELECT map_filter(map(1, 0, 2, 2, 3, -1), (k, v) -> k > v);
-- {1:0,3:-1}
map_zip_with
Supported.
map_zip_with(map1, map2, function): Merges two given maps into a single map by applying function to the pair of values with the same key. For keys only presented in one map, NULL will be passed as the value for the missing key. If an input map contains duplicated keys, only the first entry of the duplicated key is passed into the lambda function.
Since Spark 3.0.0.
SELECT map_zip_with(map(1, 'a', 2, 'b'), map(1, 'x', 2, 'y'), (k, v1, v2) -> concat(v1, v2));
-- {1:"ax",2:"by"}
SELECT map_zip_with(map('a', 1, 'b', 2), map('b', 3, 'c', 4), (k, v1, v2) -> coalesce(v1, 0) + coalesce(v2, 0));
-- {"a":1,"b":5,"c":4}
reduce
Supported.
reduce(expr, start, merge, finish): Applies a binary operator to an initial state and all elements in the array, and reduces this to a single state. The final state is converted into the final result by applying a finish function.
Since Spark 3.4.0.
SELECT reduce(array(1, 2, 3), 0, (acc, x) -> acc + x);
-- 6
SELECT reduce(array(1, 2, 3), 0, (acc, x) -> acc + x, acc -> acc * 10);
-- 60
transform
Supported.
transform(expr, func): Transforms elements in an array using the function.
Since Spark 2.4.0.
SELECT transform(array(1, 2, 3), x -> x + 1);
-- [2,3,4]
SELECT transform(array(1, 2, 3), (x, i) -> x + i);
-- [1,3,5]
transform_keys
Supported.
transform_keys(expr, func): Transforms elements in a map using the function.
Since Spark 3.0.0.
SELECT transform_keys(map_from_arrays(array(1, 2, 3), array(1, 2, 3)), (k, v) -> k + 1);
-- {2:1,3:2,4:3}
SELECT transform_keys(map_from_arrays(array(1, 2, 3), array(1, 2, 3)), (k, v) -> k + v);
-- {2:1,4:2,6:3}
transform_values
Supported.
transform_values(expr, func): Transforms values in the map using the function.
Since Spark 3.0.0.
SELECT transform_values(map_from_arrays(array(1, 2, 3), array(1, 2, 3)), (k, v) -> v + 1);
-- {1:2,2:3,3:4}
SELECT transform_values(map_from_arrays(array(1, 2, 3), array(1, 2, 3)), (k, v) -> k + v);
-- {1:2,2:4,3:6}
zip_with
Supported. Results match Spark; the auto-generated result column name can differ in some examples.
zip_with(left, right, func): Merges the two given arrays, element-wise, into a single array using function. If one array is shorter, nulls are appended at the end to match the length of the longer array, before applying function.
Since Spark 2.4.0.
SELECT zip_with(array(1, 2, 3), array('a', 'b', 'c'), (x, y) -> (y, x));
-- [{"y":"a","x":1},{"y":"b","x":2},{"y":"c","x":3}]
SELECT zip_with(array(1, 2), array(3, 4), (x, y) -> x + y);
-- [4,6]
SELECT zip_with(array('a', 'b', 'c'), array('d', 'e', 'f'), (x, y) -> concat(x, y));
-- ["ad","be","cf"]
Vector
vector_avg · vector_cosine_similarity · vector_inner_product · vector_l2_distance · vector_norm · vector_normalize · vector_sum
vector_avg
Supported. New in Spark 4.2.0.
vector_avg(array): Returns the element-wise mean of float vectors in a group. All vectors must have the same dimension.
SELECT vector_avg(col) FROM VALUES (array(1.0F, 2.0F)), (array(3.0F, 4.0F)) AS tab(col);
-- [2.0,3.0]
vector_cosine_similarity
Supported. New in Spark 4.2.0.
vector_cosine_similarity(array1, array2): Returns the cosine similarity between two float vectors. The vectors must have the same dimension.
SELECT vector_cosine_similarity(array(1.0F, 2.0F, 3.0F), array(4.0F, 5.0F, 6.0F));
-- 0.9746319
vector_inner_product
Supported. New in Spark 4.2.0.
vector_inner_product(array1, array2): Returns the inner product (dot product) between two float vectors. The vectors must have the same dimension.
SELECT vector_inner_product(array(1.0F, 2.0F, 3.0F), array(4.0F, 5.0F, 6.0F));
-- 32.0
vector_l2_distance
Supported. New in Spark 4.2.0.
vector_l2_distance(array1, array2): Returns the Euclidean (L2) distance between two float vectors. The vectors must have the same dimension.
SELECT vector_l2_distance(array(1.0F, 2.0F, 3.0F), array(4.0F, 5.0F, 6.0F));
-- 5.196152
vector_norm
Supported. New in Spark 4.2.0. Results match Spark; the auto-generated result column name can differ in some examples.
vector_norm(vector, degree): Returns the Lp norm of a float vector using the specified degree. Degree defaults to 2.0 (Euclidean norm) if unspecified. Supported values: 1.0 (L1 norm), 2.0 (L2 norm), float('inf') (infinity norm).
SELECT vector_norm(array(3.0F, 4.0F), 2.0F);
-- 5.0
SELECT vector_norm(array(3.0F, 4.0F), 1.0F);
-- 7.0
SELECT vector_norm(array(3.0F, 4.0F), float('inf'));
-- 4.0
vector_normalize
Supported. New in Spark 4.2.0. Results match Spark; the auto-generated result column name can differ in some examples.
vector_normalize(vector, degree): Normalizes a float vector to unit length using the specified norm degree. Degree defaults to 2.0 (Euclidean norm) if unspecified. Supported values: 1.0 (L1 norm), 2.0 (L2 norm), float('inf') (infinity norm).
SELECT vector_normalize(array(3.0F, 4.0F), 2.0F);
-- [0.6,0.8]
SELECT vector_normalize(array(3.0F, 4.0F), 1.0F);
-- [0.42857143,0.5714286]
SELECT vector_normalize(array(3.0F, 4.0F), float('inf'));
-- [0.75,1.0]
vector_sum
Supported. New in Spark 4.2.0.
vector_sum(array): Returns the element-wise sum of float vectors in a group. All vectors must have the same dimension.
SELECT vector_sum(col) FROM VALUES (array(1.0F, 2.0F)), (array(3.0F, 4.0F)) AS tab(col);
-- [4.0,6.0]
Function names, usage text, argument descriptions and examples are from the Apache Spark SQL built-in functions documentation, Copyright The Apache Software Foundation, licensed under the Apache License, Version 2.0. Apache Spark is a trademark of the Apache Software Foundation. Sail status and notes are from Databasin.