Guides: Business Analytics Guide: Work with data

Search the Business FAQ

Quick start

Orient yourself to research methods and software:

SAGE Research Methods
Research Methods is a tool created to help researchers, faculty and students with their research projects. Users can explore methods concepts to help them design research projects, understand particular methods or identify a new method, conduct their research, and write up their findings. Since SAGE Research Methods focuses on methodology rather than disciplines, it can be used across the social sciences, health sciences, and other areas of research.

DataQuest
Free with Penn registration. Self-paced online learning platform with interactive courses. Lessons, skill paths, and career paths cover disciplines such as data visualization, machine learning, and data scraping with tools including Python, R, PowerBI, SQL, and Tableau.

Computing infrastructure

Can't run the software on your machine? Run it online as if using a desktop PC.

Need a computer? Use physical computer labs to run specialized software in person.

Penn Libraries computer workstations
GSE Stat Lab
SAS, Stata, SPSS, ATLAS.ti. Available to GSE students taking Advanced Quantitative courses.
MPH Program Student Lounge and Research Lab
Located in Room 147 in the Anatomy-Chemistry building, the new MPH Student Research Lab has several computers which provide students with access to ArcGIS, Nvivo10, STATA and SAS.
PennDesign Computer Lab Software List
These labs are open only to those enrolled in PennDesign Courses.
Penn Nursing Student Computer Lab
Room 201 Claire M. Fagin Hall. Has SAS, Stata, SPSS.
SAS Computer Labs
Lab locations and software availability for School of Arts and Sciences.
SEAS Computing Labs (CETS)
Lab locations and software availability.
Wharton Lab Software
Available to students with Wharton accounts.

Are your procedures running for weeks? Offload processing tasks to dedicated servers.

Penn HPC
All Penn faculty, staff, and students as well as individuals from other non-profit organizations may access HPC services hosted by PSOM. You can see the fee structure under The Basics.
SAS HPC
SAS faculty and staff may access the school's General Purpose Cluster, explore the possibility of a private computing cluster, or share the Social Sciences Computing servers.
SEAS HPC
SEAS faculty have purchased MP Clusters, EP Grids, and specialized compute resources.
Wharton HPC
Wharton faculty, faculty research assistants, faculty co-authors, and doctoral candidates may apply for access.
WRDS Cloud
Register to access and analyze WRDS data online. Supports Python, R, Stata, and SAS.

Software tools

Beautiful Soup (Python library)
Use this parser to read XML or HTML content from sources such as APIs or web sites.
Microsoft Excel
Use a graphical interface to enter, view, and edit tabular data. The software emphasizes displaying data and formulaic calculations in addition to supporting visualization, analysis, and automation.
Qualtrics
Create and host surveys using this robust web platform.
more...less...
The linked Qualtrics license is available to all current Penn students, staff, and faculty. Your school may offer accounts with additional features.
SQL
Access and handle data stored in relational databases using this common query language.
more...less...
Many variations of the SQL language and software exist. One online editor is linked.

Microsoft Excel
Use a graphical interface to enter, view, and edit tabular data. The software emphasizes displaying data and formulaic calculations in addition to supporting visualization, analysis, and automation.
OpenRefine
A powerful tool for working with (cleaning, transforming) messy data.
tidyr (R package)
Tools for cleaning data efficiently in R.

HuggingFace
Machine learning models, datasets, applications, and community
JMP
Use a graphical interface to run statistical procedures in this commercial software package common in quality control and applied engineering environments.
Jupyter Notebook / JupyterLab
Make your code and output more flexible and readable with these web applications.
more...less...
You must install a compatible programming language (e.g.: Python) in order to use that language via Jupyter applications.
Python
Many researchers use this free, general-purpose programming language for text analysis and machine learning. Community-developed libraries expand support for these tasks and more.
more...less...
Often used with Jupyter Notebook.
R
Perform statistical operations and more with this free programming language originally developed for statisticians. Community-developed packages improve support for machine-learning, visualization, publishing, and more.
more...less...
Often used with RStudio or Jupyter Notebook
RStudio
Makes using R (and other languages) easier by adding a graphical interface and other convenient features.
more...less...
You must install R in order to run R with RStudio.
SAS
Program statistical operations with this commercial software package favored by government and other legacy labs.
SPSS
Edit survey data features and run statistical analyses using a programming or graphical interface. Features emphasize social science applications.
Stata
Program statistical operations with this commercial software package favored by economists.

You can install several free applications--including Python, Jupyter, and RStudio--though the one-stop Anaconda utility.

ggplot2 (R package)
Create and fine-tune data visualizations in R.
Microsoft Excel
Use a graphical interface to enter, view, and edit tabular data. The software emphasizes displaying data and formulaic calculations in addition to supporting visualization, analysis, and automation.
Tableau for Students
Free version of Tableau (data visualization software) designed for educational use that offers more functionality than Tableau Public.

Blackbox
Conversational code suggestions and code analysis
Codeium
Chat-based and inline code suggestions, including refactoring
GitHub Copilot
Conversational or terminal-integrated code suggestions

Methods

SAGE Research Methods
Research Methods is a tool created to help researchers, faculty and students with their research projects. Users can explore methods concepts to help them design research projects, understand particular methods or identify a new method, conduct their research, and write up their findings. Since SAGE Research Methods focuses on methodology rather than disciplines, it can be used across the social sciences, health sciences, and other areas of research.

DataQuest
Free with Penn registration. Self-paced online learning platform with interactive courses. Lessons, skill paths, and career paths cover disciplines such as data visualization, machine learning, and data scraping with tools including Python, R, PowerBI, SQL, and Tableau.

Company identifier conversion

Look up a list of companies by name or ID with the S&P Capital IQ Pro Identifier Converter. The tool will try to match your submissions to S&P's companies database and return as many as five potential matches per submitted name/ID.

Requirements

Your S&P Capital IQ Pro account
Excel for Windows with the CapIQ Pro add-in
At least one accepted company identifier

Known accepted company identifiers:
(expand/collapse)

Company name, including an approximate or related name
MI Key
SPCIQ ID
Ticker symbol
ISIN
Trading Item ID / Ticker number
SEDOL
CUSIP (9 digits only)
Legal Entity ID (LEI)

Download the template

Open Excel.
From the ribbon menu, select S&P Cap IQ Pro > S&P Capital IQ Pro Data > Templates.
In the Template Library window that opens, use the Search Templates field to search the term identifier converter.
Use the Identifier Converter link to download the template file SPG_IdentifierConverter_v1.xlsm.

Run the converter

Open PG_IdentifierConverter_v1.xlsm in Excel. You may need to Enable Content and/or unblock macros if Excel displays warning messages.
Use the CLEAR DATA button to initialize the worksheet.
List your company identifiers (one per row) under Search Company Input (cell C40 and below).
Use the SEARCH RESULTS button to retrieve results.

Edit the template to return other entity-level values

(expand/collapse)

You can change the template’s behavior to return different fields from Capital IQ Pro.

Unhide columns O through BM.
Starting in cells AC67 through AH67, replace the field codes with other valid CapIQ Pro field codes.
Use the SEARCH RESULTS button to retrieve results.

Valid CapIQ Pro field codes include, but are not limited to,

SPCIQ ID*: SP_CIQ_ID
Ticker-Exchange*: SP_TICKER_EXCHANGE
Country / Region Name*: SP_COUNTRY_NAME
MI Key: SP_COMPANY_ID
ISIN: SP_ISIN
Trading Item ID: SP_TRADING_ITEM
Issue SEDOL: SP_SEDOL
Issue CUSIP (9 digits): SP_CUSIP
Legal Entity ID (LEI): SP_LEI
Tax Identification Number (TIN): SP_TAX_ID

*Default field

To find more codes,

Open Formula Builder.
Under DATA ITEMS, expand Company Details > Company Identification.
Select a field name.
Look under SELECTED FORMULAS to see its code.

Look up a list of companies (or people) by name with the SPCIQ Identifier Converter.

Submit name (includes related and approximate matches), ticker*, CIQ-ID, ISIN*, or CUSIP (6, 8, or 9 digits)*
* Companies lookup only
Return as many as five results per identifier for name, CIQ-ID, and more
Edit results to return additional fields (e.g.: GVKEY; ISIN; CIK)

Requirements:

Your S&P Capital IQ (CIQ) account
Excel (for Windows) with the CIQ Excel plug-in
At least one identifier (see above) for a company or person

Download the template:

Open Excel.
From the menu ribbon, select S&P Capital IQ > (Data) Templates > Get/Update Templates.
Expand and select SPCIQ Excel Plug-In Tools > SPCIQ Identifier Converter.xlsm.
Download.

Run the converter:

Select S&P Capital IQ > (Data) Templates > SPCIQ Excel Plug-In Tools > SPCIQ Identifier Converter to open the template.
Enter one or more identifiers (one per row) under Search Item Input (column AH).
Note: Start CUSIPs with ' to retain leading zeros.
Search.

Edit the template to return other entity-level values:

Run the converter as above.
Unhide columns to the left of column AG.
In the formulas (columns M through AF), replace each mention of a given IQ_... variable with your preferred CIQ variable (e.g.: IQ_GVKEY; IQ_ISIN; IQ_CIK).

Note: Do not Search again. The search button macro will reset your edited functions.

SPCIQ Identifier Converter showing a function in the process of conversion to a different identifier

Convert common company and/or security identifiers with WRDS (Wharton Research Data Services).

Explore the Linking Suite, search WRDS for converter or linking table, or start with highlights:

CUSIP: under Get Data > Compustat - Capital IQ > Tools > CUSIP Converter
- Submit 8 or 9 digit CUSIP
- Return 6, 8, or 9 digit CUSIP
CRSP/Compustat: under Get Data > CRSP > Annual Update > CRSP/Compustat Merged > Compustat CRSP Link; parallel tool under Quarterly Update; parallel tool under Monthly Update
- Submit GVKEY, PERMNO, PERMCO, ticker, CUSIP, CIK, or industry classification
- Return PERMNO, PERMCO, company name, ticker, CUSIP, CIK, and more
Exchange code: under Get Data > CRSP > Annual Update > Tools > Exchange Code Search; parallel tool under Quarterly Update
- Submit stock exchange
- Return PERMCO, CUSIP, ticker, company name, share code, and more
Historical Company Names of SEC Filers: under Get Data > SEC Analytics Suite by WRDS > SEC Linking Tables > Historical Company Names)
- Submit ticker, CIK, GVKEY, or CUSIP
- Return historical company name, ticker, GVKEY, CUSIP, and more
Court case (bankruptcy, civil, or criminal): under Get Data > Federal Judicial Center > WRDS Linking > FJC Compustat-CIK Link...
- Submit docket number, case key (bankruptcy only), GVKEY, or CIK
- Return docket number, case key (bankruptcy only), company name (civil only), GVKEY, or CIK, and more

PitchBook can accept your list of firms, compare features you provide with data in PitchBook's database, and suggest firm matches.

Log in to PitchBook
Along the left, choose My PitchBook > Upload a List
Follow the provided steps
Use your results in PitchBook's Advanced Search, or export to a CSV file to retain PitchBook firm identifiers for reference (e.g.: in WRDS)

Business & Data Analysis Librarian

Kevin Thomas

He/Him/His

Email Me

Subjects: Statistics

Business Analytics Guide