Shared Access Management pools#

Shared Access Management (SAM) pools enable a single set of credentials to be shared between client side threads to leverage multiple threads of parallelism on the 1010data Insights Platform.

Creating a new py1010.Session object for a SAM pool user is similar to creating a single-user Session object.

testSession=py1010.Session(gateway, owner, pwd, py1010.POSSESS, group)

The SAM pool user session takes two different parameters. Instead of the individual user name, this Session object takes the owner parameter. The owner parameter is the group owner’s user name. The group parameter is the group ID. The SAM pool Session constructor takes the following parameter variables:

gateway - a 1010data Insights Platform URI
group - a group ID
owner - the group owner ID
password - the group owner password

If your users are using a SAM pool to log in, they will not have their own platform IDs. The users share a single set of credentials to log into the platform. A SAM pool has a number of IDs to use, but it is not limitless. You may have to account for cases where peak utilization creates a situation where more threads are trying to access the platform than your SAM pool is configured to handle.

The Python SDK is designed to queue during Session object construction when a SAM pool is fully utilized. During this queuing time the thread trying to construct a new Session object retries the request every ten seconds.

To ensure that your SAM pool is fully utilized, clean up sessions as soon as your thread is done to prevent a full pool of IDs where all of the threads of database concurrency are in use at the same time. For more information, see Cleaning up a session.

In this example, two threads construct two separate Session objects and log in with a SAM pool. The application retrieves data from two different tables and downloads them into two separate files.

import py1010
import threading
import sys

gateway = "https://www2.1010data.com/cgi-bin/gw"
group = "[GROUP_ID]"
owner = "[GROUP_OWNER_USERNAME]"
pwd = "[GROUP_OWNER_PASSWORD]"
tablename = "pub.demo.weather.stations"

def runQuery(table, ops, outfilename):
 try:
     with py1010.Session(gateway, owner, pwd, py1010.POSSESS,
                         group) as session:
         print("Logged in as {}".format(session.username))
         query = session.query(table, ops)
         query.run()
         with open(outfilename, "w") as outfile:
             print("Starting to write to " + outfilename)
             for r in query.rows:
                 outfile.write("\t".join([str(x) for x in r]) + "\n")
 except py1010.TentenException as e:
     print("Error: {}".format(e))

t1=threading.Thread(target=runQuery,
                 args=("pub.public_data.census.acs.all_acs_processed_tables",
                       '<sel value="(i_<100000)"/>', "Output1.txt"))
t2=threading.Thread(target=runQuery,
                 args=("pub.public_data.census.acs.reference.column_names",
                       '<sel value="(i_<100000)"/>', "Output2.txt"))
t1.start()
t2.start()

t1.join()
t2.join()

Note

[GROUP_ID], [GROUP_OWNER_USERNAME], and [GROUP_OWNER_PASSWORD] are placeholders for valid Insights Platform group ID, owner name, and password.

`flushpool` (Flush a SAM pool)#

py1010.flushpool() flushes the SAM pool. This ensures that user IDs returned from this point on are fresh and not already logged in.

You may want to periodically request your IDs to start new sessions due to memory overconsumption. You may also want to request IDs to update their view of the database if a table or set of tables have been updated. In these cases, you can use flushpool.

flushpool terminates all sessions once they have been released back into the pool. It will not terminate any allocated or busy sessions.

The SAM pool group owner and the admin of the owner’s company can call flushpool. This command does not actually log in the requesting ID.

The flushpool function takes the following parameters:

The URL of the 1010 gateway (url)
Group owner name (owner)
Group owner password (password)
Group ID (group)
Path to a log file (logfile) - The default value is None.

py1010.flushpool(gateway, owner, password, group, logfile, mode)

`resetpool` (Reset a SAM pool)#

You can release all of the user IDs in a specific SAM pool using the py1010.resetpool() function.

Note

This function is intended for development and debugging.

The resetpool function takes the following parameters:

The URL of the 1010 gateway (url)
Group owner name (owner)
Group owner password (password)
Group ID (group)
Path to a log file (logfile) - The default value is None.

py1010.resetpool(gateway, owner, password, group, logfile)

This function will disrupt any user who may be working with a SAM pool ID at the time.

`warmpool` (Warm a SAM pool)#

One way to reduce the amount of time it takes to retrieve results from the 1010data Insights Platform is to “warm” the SAM pool by having pool IDs logged in ahead of time.

An ID from a SAM pool that was warmed is already logged into the Insights Platform. The py1010.warmpool() function logs in available SAM pool user IDs, which can speed the retrieval of results for users. To this end, you can also provide queries to run on the warmed IDs when they log in.

The warmpool function takes the following parameters:

The URL of the 1010 gateway (url)
Group owner name (owner)
Group owner password (password)
Group ID (group)
List of BaseQuery objects - BaseQuery objects take a table name and 1010data Macro Language query, but are different from Query objects because they are not associated with a Session object. The default value is None.
Path to a log file (logfile) - The default value is None.

Example without queries or a log file path and with them:

py1010.warmpool(gateway, owner, password, group)

py1010.warmpool(gateway, owner, password, group, queries=None, logfile=None)

You can run warmpool without logging in.

warmpool returns the number of IDs warmed.

After the query results are retrieved, the used IDs should be released back into the pool of available IDs. For more information, see Cleaning up a session. These released IDs are not re-warmed automatically. When they are used again, the ID will have to be logged in.

The example below uses warmpool to warm the SAM pool and BaseQuery to provide a query:

#!/usr/bin/env python
import py1010

url = "https://www2.1010data.com/prod-latest/gw"
owner = "USERNAME"
group = "GROUP"
password = "PASSWORD"

queries = [
  py1010.BaseQuery("pub.demo.weather.stations", ""),
  py1010.BaseQuery("pub.demo.weather.hourly90",
                      '''<sel value="(id=3103)"/>''')
  ]

warmed=py1010.warmpool(url, owner, password, group,
                                queries=queries,
                                logfile="FOOO.log");

print("Warmed: %d"%warmed)

Note

USERNAME, GROUP, and PASSWORD are placeholders for valid Insights Platform group owner name. group name, and password.

Shared Access Management pools#

flushpool (Flush a SAM pool)#

resetpool (Reset a SAM pool)#

warmpool (Warm a SAM pool)#

`flushpool` (Flush a SAM pool)#

`resetpool` (Reset a SAM pool)#

`warmpool` (Warm a SAM pool)#