tag:blogger.com,1999:blog-5261056907132640554.post5528854945084445195..comments2015-11-09T12:02:39.938-08:00Comments on bizo developer blog: 4 tips from the trenches of Amazon Elastic MapReduce and Hivelarry ogrodnekhttp://www.blogger.com/profile/01105034385285773975noreply@blogger.comBlogger4125tag:blogger.com,1999:blog-5261056907132640554.post-65889213287967081732013-10-08T08:45:56.751-07:002013-10-08T08:45:56.751-07:00You can access the members of a map using square b...You can access the members of a map using square brackets. eg,<br /><br />select d["timestamp"], d["id"] from sample_data_2011_12 ;<br />Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-5261056907132640554.post-57168206091501497782013-10-08T04:50:09.720-07:002013-10-08T04:50:09.720-07:00In the past, I have explicitly specified column na...In the past, I have explicitly specified column names & data types when creating hive table for importing access logs. If you create a table with the following statement as mentioned in tip 2, how do you query for individual columns?<br /><br />create external table sample_data_2011_12(d map)<br />g3t r00thttps://www.blogger.com/profile/09860128909809442458noreply@blogger.comtag:blogger.com,1999:blog-5261056907132640554.post-82442528299824575372011-12-19T08:36:27.534-08:002011-12-19T08:36:27.534-08:00We try to keep our report runtimes under a few hou...We try to keep our report runtimes under a few hours; in our experience, it's often cheaper to increase the number of machines (especially spot instances) then to let a smaller number of instances run for many hours, so throwing more instances at the problem is win-win.<br /><br />It's actually uncommon for us to use the monthly view; our strategy for monthly reports is usually to do daily rollups of some sort, then aggregate the rollups at the end of the month. I haven't actually run the numbers in a while, but my off-the-cuff guess is that using an entire month of data be somewhere in the 10-100 terabytes range, depending on which services are involved.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-5261056907132640554.post-16791834482400145472011-12-19T08:16:47.951-08:002011-12-19T08:16:47.951-08:00Some really great tips, thanks for posting. What ...Some really great tips, thanks for posting. What kind of reporting latency are you seeing, for example in your 2011-12 monthly view (and how much data storage is required).<br /><br />JamesJames Holcombhttps://www.blogger.com/profile/05377825007589191412noreply@blogger.com