2.5 Raw Data Ingestion with INFILE and INPUT Statements

2.5 Raw Data Ingestion with INFILE and INPUT Statements

While PROC IMPORT provides quick automatic file ingestion, SAS Base programmers must master the INFILE and INPUT statements inside the DATA step for complete control over raw data parsing, complex informats, line-hold specifiers, and custom record structures.

---

1. The INFILE Statement & Essential Options

The INFILE statement specifies the external raw data file to be read by the INPUT statement in a DATA step.

data work.raw_ingest;
    infile "C:\\Data\\raw_payroll.txt" dlm=',' dsd missover firstobs=2 obs=100;
    input employee_id : $8. hire_date : mmddyy10. salary : dollar10.2;
run;

Key INFILE Statement Options

---

2. Line Boundary Handling: FLOWOVER vs. MISSOVER vs. TRUNCOVER

Understanding how SAS handles short records (records with fewer fields than variables in the INPUT statement) is critical for the exam:

• FLOWOVER (the default): If an INPUT statement reaches the end of a raw record before populating all variables, SAS automatically flows over to the next raw data line to read values for the remaining variables, consuming multiple physical lines per observation. The option is spelled FLOWOVER, not FLOW, and you rarely code it explicitly because it is already in effect.
• MISSOVER: If an INPUT statement reaches the end of a raw record, SAS stops reading that record and sets all remaining unread variables to missing.
• TRUNCOVER: Essential for formatted/column input. If a raw field is shorter than the informat width or truncated by the end of the line, TRUNCOVER reads whatever characters exist up to the line boundary into the PDV, rather than setting the variable to missing or jumping to the next record.

/* Correct handling of short variable-length records */
data work.phone_directory;
    infile "C:\\Data\\contacts.txt" missover;
    input id $ name $ phone $ email $;
run;

---

3. INPUT Statement Styles

SAS supports four distinct styles of input in the INPUT statement:

1. List Input

Variables are separated by spaces or delimiters. Values are read sequentially.

input emp_id $ age department $ salary;

2. Column Input

Field locations are explicitly specified by column start and end positions. Spaces do not act as delimiters, allowing embedded blanks in string values.

input name $ 1-20 title $ 21-35 age 36-37;

3. Formatted Input

Combines pointer controls (@n, +n) with SAS informats (instructions on how to read raw data formats into standard numeric or character values).

input @1 emp_id $5. @7 hire_date mmddyy10. @18 salary dollar10.2;

4. Modified List Input

Uses informat modifiers (: or ~) alongside standard list input:

• : Modifier: Instructs SAS to read past the raw data value until it encounters a delimiter, applying the specified informat width without being constrained by fixed column boundaries.
• ~ Modifier: Preserves quotes in string values when used with DSD.

/* Colon modifier allows reading formatted dates of varying field lengths */
input name : $20. start_date : date9. annual_pay : comma12.2;

---

4. Pointer Controls & Line-Hold Specifiers (@ and @@)

Pointer controls direct SAS where to read in the input buffer:

• @n: Absolute column pointer (moves pointer directly to column n).
• +n: Relative column pointer (moves pointer forward n columns).
• /: Line pointer control (advances pointer to column 1 of the next raw line).

Trailing Line-Hold Specifiers

+-------------------------------------------------------------------------+
|                        TRAILING LINE-HOLD SPECIFIERS                   |
+-------------------------------------------------------------------------+
|  Single Trailing @  | Holds line in buffer for current DATA step iter.  |
|  Double Trailing @@ | Holds line in buffer across multiple iterations.  |
+-------------------------------------------------------------------------+

Single Trailing @

Holds a raw record in the input buffer so that subsequent INPUT statements within the same iteration of the DATA step can execute conditional logic before reading the rest of the record.

data work.managers;
    infile "C:\\Data\\staff.txt";
    input type $ 1-1 @; /* Read type, hold record in buffer */
    if type = 'M' then do;
        input id $ 3-6 name $ 8-25 bonus dollar8.2;
        output;
    end;
run;

Double Trailing @@

Holds a raw record in the input buffer across multiple iterations of the DATA step. Used when a single line of raw data contains multiple observations.

/* Single line contains 3 observations: 101 25 102 30 103 28 */
data work.patient_ages;
    infile datalines;
    input patient_id age @@; /* Holds line across iterations until empty */
    datalines;
101 25 102 30 103 28
104 42 105 39
;
run;