How to download random pictures

Thanks Rhode Island Narragansett School System for the awesome avatar for this post!

As CloudBerry Lab engineer sometimes I run stress tests for Backup and Explorer product portfolio and it requires a lot of data. The below script helps me to fetch google through GCP APIs for images with certain content. See how I get thousand giraffes for in my local folder.

All you need:


require 'google_custom_search_api'
require "open-uri"
require 'json'

GOOGLE_API_KEY = "GOOGLE_API_KEY"
GOOGLE_SEARCH_CX = "GOOGLE_SEARCH_CX"

options = {}
options[:searchType] = "image"
page = 1

10.times do |page|
  options[:page] = page
  results = GoogleCustomSearchApi.search("giraffe", options)
  results["items"].each do |item|
    download = open(item["link"])
    puts item["link"]
    IO.copy_stream(download, "images/#{download.base_uri.to_s.split('/')[-1]}")
  end
  # page++
  # puts JSON.pretty_generate results
end


You can convert this to Python and use along with serverless architecture (for instance Lambda in AWS). Should be fun!


In short, this is about:
#ruby
#cloudberrylab

Start discussion:
Related articles:
120 ec2 user data script example preview
Launching AWS EC2 can be supplied by something what is known as "user data" and it helps to have things ready before you access it. ... more
89 sinatra if you need lighter preview
Sinatra ruby is the light way to work with http methods. If you consider your App to be really light and just endpoints, think "not-Rails". ... more
over 1 year#sinatra #ruby
83 cloudberry lab for mac os x 1 8 is now live whoohoo preview
Cloudberry 1.8 for Mac OS X is now live! Impressive list of features and storage platforms support! S3 compatible, new UI and etc. ... more
over 1 year#cloudberrylab #mac
113 grant fullaccess to all mailboxes in exchange 2010 preview
Since FullAccess is disabled even for Administrator's account in Exchange (including 2010) CloudBerry Backup for Exchange can't restore using Administrator's username and password pair for item level restore of any mailbox. Let's fix it. ... more
105 conveyoring files for backup cloudberry backup with pre post scripts preview
Working with customers is great experience and you always learn or invent something new. I've been working on the case with files portioning ... more
121 aws ec2 user data script windows preview
How to pre-deploy App using user data script for Windows OS with powershell powered script in EC2? Example inside. ... more
112 how to follow the 3 2 1 rule with cloudberry backup preview
In this article you'll learn how to protect your data against hardware and software failures according to the 3-2-1 rule. ... more
95 the nokogiri version of scraping data preview
One of the evening I spend on parsing xml content with 1000+ pages. grep, awk, sed were awesome, but the head were getting bigger with massive number of regex's, so nokogiri made it easier. ... more
about 1 year#nokogiri #ruby
86 call recording asterisk preview
Asterisk IP PBX comes with embedded MixMonitor, that allows us to to record and place those recordings somewhere. ... more
101 aws ec2 linux backup preview
Amazon Web Services (AWS) has its own AWS Linux AMI (also free tier). There are several reasons you want to use this AMI for your projects. ... more