![]() ![]() ![]() Redshift Spectrum configuration is similar. CREATE EXTERNAL TABLE spectrum_schema_vs.unique_users_expĪS SELECT distinct * FROM public. The AWS Region in which the Amazon Redshift cluster and Amazon S3 bucket are located must be the same. This method is different from COPY command or UNLOAD command. You can unload the Amazon Redshift tables to S3 folder by using CREATE EXTERNAL TABE AS command.įor example, following command will unload the unique_users table to an external S3 location. (1 row) Export Redshift Tables Using External Tables Amazon Redshift Spectrum allows users to create external tables, which reference data stored in Amazon S3, allowing transformation of large data sets. (5/TB x 1 TB 5) If you compress your file and convert it to a columnar format like Apache Parquet, you may see a 4:1 compression ratio and have a compressed file size of 1 TB. 493 10 31 Add a comment 1 Answer Sorted by: 10 In the first instance I prefer to use Redshift for transformations as: Development is easier, SQL rather than Spark Maintenance / monitoring is easier Infrastructure costs are lower assuming you can run during 'off-peak' times. Spectrum_schema_vs | ext_users | s3://vithal/data Redshift Spectrum has to scan the entire file, but since it is one-fourth the size, you pay one-fourth the cost, or 5. You can query the svv_external_tables Redshift system table to display the external tables available on AWS Redshift.įor example, dev=# select schemaname, tablename, location from svv_external_tables ![]() Access External Tables in Redshift List External Tables Available on Amazon Redshift ![]()
0 Comments
Leave a Reply. |