Preprocess project source files for preparing API parameter compatibility issue detection and repair
More...

Functions
def	addDictAll (projPath, projName, filePath, copyRoot, runFileLst, libName, runPath, runCommand, pcresolveLookup=None)
	Code instrumentation for all API calls within a project source file 项目源文件所有API调用代码插桩 More...

def	addDictSingle (callAPI, filePath, callKey)
	Code instrumentation for a single API call within a source file 源文件单个API调用代码插桩 More...

def	codeProcess (projPath, runCommand, runPath, libName, workspace, pcresolveLookup=None)
	Code processing 代码预处理 More...

def	convertLocalVar (filePath, libName)
	Convert the ListComp and DictComp statements to variable assignment statements 将列表推导式listComp和字典推导式DictComp转换为变量赋值表达式 More...

def	convertTabsToSpaces (directory)
	Convert tabs to spaces in all Python files within a directory 将目录下所有Python文件中的制表符转换为空格 More...

def	countBracket (s)
	Count the number of different types of brackets ((), [], {}) in a string 计算字符串中各类括号((), [], {})的个数 More...

def	countSpace (s)
	Count the number of spaces at the beginning of a string 计算字符串的前面有多少个空格 More...

def	expandConditionalReturn (filePath)
	Expand the single-line conditional return statement into a multi-line if-else structure 展开单行条件return语句为多行if-else结构 More...

def	extractDecorator (root)
	Extract decorator API calls 抽取项目中第三方库装饰器调用 More...

def	findAssignCall (root)
	Find assignment call statements 获取赋值调用语句 More...

def	getAssignReceiverExpr (root, source, aliasName, lineno)
	Resolve the source expression assigned to a receiver alias 还原调用者别名在当前作用域内的赋值来源表达式 More...

def	getDecoratorInsertIndex (codeLst, index)
	Get decorator block start index 获取装饰器块起始行号 More...

def	getDictVar (root, ansLst)
	Get the local variable in the dictionary comprehension 获取字典推导式中的局部变量 More...

def	getImportLine (codeLst)
	Determine the instrumentation line for import statements 确定import语句插桩行 More...

def	getLibImportLst (projPath, libName)
	Get all lib-related import statements 获取所有第三方库相关的import语句 More...

def	getListVar (root, ansLst)
	Get the local variable in the list comprehension 获取列表推导式中的局部变量 More...

def	getSelfReceiverExpr (root, lineno, methodName)
	Resolve the visible base-class expression for an inherited self receiver 还原继承场景中self调用者可见的基类表达式 More...

def	handleRunFile (file, runPath, runCommand)
	Code instrumentation for project run file 项目运行文件代码插桩 More...

def	ignore_sym_links (directory, files)
	Determine the soft link files in a directory 确定文件夹中的软链接文件 More...

def	modifyFromImport (filePath, importStatement)
	Save import statement to source file 将import语句保存到源码中 More...

def	obtainDef (sourcePath)
	Extract class and function definitions from a source file 从源文件中提取类和函数定义 More...

def	oneLine (filePath)
	Convert the multi-line parameter calls in the code into a single line to facilitate insertion into dictionary statements 把代码中换行写的参数调用，合并成一行，目的是便于插入字典语句 More...

def	saveConstantAssign (astNode, constantVar, nonConstantVar, decoratorLst)
	Save assignment statement with constant values 保存常量赋值语句 More...

def	saveStructure (projPath, libName)
	Save structure information of the project 保存项目的结构信息 More...

def	scriptPath (scriptName)
	Get source script path from PCART repository 获取PCART仓库中的辅助脚本路径 More...

def	writeRecordValue (filePath, pklRelPath, useCallsiteName)
	Write recordValue.py used by instrumented project files 写入插桩文件共享的recordValue.py. More...

Detailed Description

Preprocess project source files for preparing API parameter compatibility issue detection and repair

Prepares project source files for analysis: flattens control flow (list/dict comprehensions, conditional returns), normalizes formatting (tabs→spaces, single-line calls), and injects runtime instrumentation (recordValue.py) around every target-library API call to capture receiver objects and parameter values at runtime. Also manages Copy/ and Dynamic/ project copies. 准备项目源文件用于分析：展平控制流（列表/字典推导式、条件返回）、格式化归一化（tabs→空格、多行调用合并为单行），并在每个目标库API调用周围注入运行时插桩（recordValue.py）以捕获接收者对象和参数值。同时管理Copy/和Dynamic/项目副本。

Function Documentation

◆ addDictAll()

def preprocess.addDictAll	(	projPath,
		projName,
		filePath,
		copyRoot,
		runFileLst,
		libName,
		runPath,
		runCommand,
		pcresolveLookup = `None`
	)

Code instrumentation for all API calls within a project source file 项目源文件所有API调用代码插桩

Parameters

projPath	The project path
projName	The project name
filePath	The project source file path
copyRoot	The runtime Copy directory
runFileLst	The list of run files
libName	The lib name
runPath	The relative path of the run file
runCommand	The run command of the project
pcresolveLookup	Optional pre-built PCResolve lookup table

Definition at line 579 of file preprocess.py.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ addDictSingle()

def preprocess.addDictSingle	(	callAPI,
		filePath,
		callKey
	)

Code instrumentation for a single API call within a source file 源文件单个API调用代码插桩

Parameters

callAPI	The API call
filePath	The source file path
callKey	Unique callsite artifact id

Definition at line 436 of file preprocess.py.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ codeProcess()

def preprocess.codeProcess	(	projPath,
		runCommand,
		runPath,
		libName,
		workspace,
		pcresolveLookup = `None`
	)

Code processing 代码预处理

Preprocessing objectives:

Modify dynamic runtime scripts for the target project
Convert tabs to spaces and merge multi-line statements for instrumentation
Inject necessary import headers 代码预处理目的：

修改一些动态运行的脚本
将用户代码的tab键用四个空格替换，再把换行写的语句集中到一行，便于插桩处理
插入一些头文件 runCommand可能是 src/run.py –config json

Parameters

projPath	The project path
runCommand	The run command of the project
runPath	The relative path of the run file
libName	The lib name
workspace	RunWorkspace object for this execution
pcresolveLookup	Optional pre-built PCResolve lookup table shared with analysis

Returns: None

Definition at line 1094 of file preprocess.py.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ convertLocalVar()

def preprocess.convertLocalVar	(	filePath,
		libName
	)

Convert the ListComp and DictComp statements to variable assignment statements 将列表推导式listComp和字典推导式DictComp转换为变量赋值表达式

Parameters

filePath	The source file path
libName	The name of the lib

Definition at line 156 of file preprocess.py.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ convertTabsToSpaces()

def preprocess.convertTabsToSpaces ( directory )

Convert tabs to spaces in all Python files within a directory 将目录下所有Python文件中的制表符转换为空格

Parameters

directory The directory path to process

Definition at line 1030 of file preprocess.py.

Here is the caller graph for this function:

◆ countBracket()

def preprocess.countBracket ( s )

Count the number of different types of brackets ((), [], {}) in a string 计算字符串中各类括号((), [], {})的个数

Parameters

s	The string

Returns: (minL, minR, midL, midR, huaL, huaR) minL, minR: the number of "(" and ")"; midL, midR: the number of "[" and "]"; huaL, huaR: the number of "{" and "}"

Definition at line 31 of file preprocess.py.

◆ countSpace()

def preprocess.countSpace ( s )

Count the number of spaces at the beginning of a string 计算字符串的前面有多少个空格

Parameters

s	The string

Returns: cntSpace The number of spaces

Definition at line 344 of file preprocess.py.

Here is the caller graph for this function:

◆ expandConditionalReturn()

def preprocess.expandConditionalReturn ( filePath )

Expand the single-line conditional return statement into a multi-line if-else structure
展开单行条件return语句为多行if-else结构

Parameters

filePath The source file path

Definition at line 82 of file preprocess.py.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ extractDecorator()

def preprocess.extractDecorator ( root )

Extract decorator API calls 抽取项目中第三方库装饰器调用

Parameters

root	The AST root of the source code file

Returns: decoratorLst The list of decorator API calls

Definition at line 391 of file preprocess.py.

Here is the caller graph for this function:

◆ findAssignCall()

def preprocess.findAssignCall ( root )

Find assignment call statements 获取赋值调用语句

Parameters

root	The AST root of the source code

Returns: assignLst List of assignment call statements

Definition at line 259 of file preprocess.py.

Here is the caller graph for this function:

◆ getAssignReceiverExpr()

def preprocess.getAssignReceiverExpr	(	root,
		source,
		aliasName,
		lineno
	)

Resolve the source expression assigned to a receiver alias 还原调用者别名在当前作用域内的赋值来源表达式

Parameters

root	The AST root of the source code
source	The original source code
aliasName	The first name of the receiver alias
lineno	The line number of the call site

Returns: expr The scoped assignment expression or None

Definition at line 276 of file preprocess.py.

Here is the caller graph for this function:

◆ getDecoratorInsertIndex()

def preprocess.getDecoratorInsertIndex	(	codeLst,
		index
	)

Get decorator block start index 获取装饰器块起始行号

Instrumentation for decorator calls must be inserted before the whole decorator block to keep Python syntax valid. 装饰器调用插桩必须插入整个装饰器块之前，避免破坏Python语法。

Parameters

codeLst	Code list read by f.readlines()
index	The matched code line index

Returns: insertIndex The safe instrumentation index

Definition at line 412 of file preprocess.py.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ getDictVar()

def preprocess.getDictVar	(	root,
		ansLst
	)

Get the local variable in the dictionary comprehension 获取字典推导式中的局部变量

For example, {k:v for k,v in {'a': 1, 'b': 2, 'c': 3}.items()} --> k, v=[(k, v) for (k, v) in {'a': 1, 'b': 2, 'c': 3}.items()][0] 比如{k:v for k,v in {'a': 1, 'b': 2, 'c': 3}.items()}转换后为k, v=[(k, v) for (k, v) in {'a': 1, 'b': 2, 'c': 3}.items()][0]

Parameters

root	The AST root of the line of code
ansLst	List of converted local variable(s) in dictionary comprehension

Definition at line 132 of file preprocess.py.

Here is the caller graph for this function:

◆ getImportLine()

def preprocess.getImportLine ( codeLst )

Determine the instrumentation line for import statements 确定import语句插桩行

Default line is 0. Avoid instrumenting import statements before future statements or between comments 默认行为0，避免在__future__前或注释块中插入import语句

Parameters

codeLst Code list read by f.readlines()

Returns: index The appropriate index for import statement instrumentation

Definition at line 363 of file preprocess.py.

Here is the caller graph for this function:

◆ getLibImportLst()

def preprocess.getLibImportLst	(	projPath,
		libName
	)

Get all lib-related import statements 获取所有第三方库相关的import语句

Parameters

projPath	The project path
libName	The project name

Returns: ansLst List of all lib-related import statements

Definition at line 999 of file preprocess.py.

Here is the caller graph for this function:

◆ getListVar()

def preprocess.getListVar	(	root,
		ansLst
	)

Get the local variable in the list comprehension 获取列表推导式中的局部变量

For example, [x**2 for x in range(10)] --> x=[x for x in range(10)][0] 比如[x**2 for x in range(10)]转换为 x=[x for x in range(10)][0]

Parameters

root	The AST root of the line of code
ansLst	List of converted local variable(s) in list comprehension

Definition at line 105 of file preprocess.py.

Here is the caller graph for this function:

◆ getSelfReceiverExpr()

def preprocess.getSelfReceiverExpr	(	root,
		lineno,
		methodName
	)

Resolve the visible base-class expression for an inherited self receiver 还原继承场景中self调用者可见的基类表达式

Parameters

root	The AST root of the source code
lineno	The line number of the call site
methodName	The self method name

Returns: expr The base-class expression or None

Definition at line 320 of file preprocess.py.

Here is the caller graph for this function:

◆ handleRunFile()

def preprocess.handleRunFile	(	file,
		runPath,
		runCommand
	)

Code instrumentation for project run file 项目运行文件代码插桩

Used during dynamic matching for single-API re-instrumentation. In addition to the current file, the run file is also instrumented so that bak_Proj run files are prepared in advance. 该函数用于动态匹配时对单个API进行插桩，除了要插桩当前文件还要对运行文件进行插桩，所以提前把bak_Proj中的运行文件处理好。

Parameters

file	The run file
runPath	The relative path of the run file
runCommand	The run command of the project

Definition at line 795 of file preprocess.py.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ ignore_sym_links()

def preprocess.ignore_sym_links	(	directory,
		files
	)

Determine the soft link files in a directory 确定文件夹中的软链接文件

Parameters

directory	The directory
files	The files in the directory

Returns: [f for f in files if os.path.islink(os.path.join(directory, f))] List of soft link files

Definition at line 988 of file preprocess.py.

◆ modifyFromImport()

def preprocess.modifyFromImport	(	filePath,
		importStatement
	)

Save import statement to source file 将import语句保存到源码中

Parameters

filePath	The source file path
importStatement	The import statement to be saved

Definition at line 832 of file preprocess.py.

Here is the caller graph for this function:

◆ obtainDef()

def preprocess.obtainDef ( sourcePath )

Extract class and function definitions from a source file 从源文件中提取类和函数定义

Parameters

sourcePath The source file path

Definition at line 811 of file preprocess.py.

◆ oneLine()

def preprocess.oneLine ( filePath )

Convert the multi-line parameter calls in the code into a single line to facilitate insertion into dictionary statements 把代码中换行写的参数调用，合并成一行，目的是便于插入字典语句

Parameters

filePath The source file path

Definition at line 66 of file preprocess.py.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ saveConstantAssign()

def preprocess.saveConstantAssign	(	astNode,
		constantVar,
		nonConstantVar,
		decoratorLst
	)

Save assignment statement with constant values 保存常量赋值语句

保留包含常量的全局赋值语句和装饰器调用相关的赋值语句，例如: Case 1:

a = 1
b = 1
c = a + b
c = func(a,b) 仅保留1，2，3行代码 Case 2:

app = Flask(name)
@app.route("/") 保留 app = Flask(name)以解决NameError

Parameters

astNode	The AST node
constantVar	Constant variable
nonConstantVar	Non-constant variable
decoratorLst	Decorator API call list

Returns: astBody The AST of the saved assignment statement; 1 for none

Definition at line 866 of file preprocess.py.

Here is the caller graph for this function:

◆ saveStructure()

def preprocess.saveStructure	(	projPath,
		libName
	)

Save structure information of the project 保存项目的结构信息

Parameters

projPath	The project path
libName	The lib name

Definition at line 921 of file preprocess.py.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ scriptPath()

def preprocess.scriptPath ( scriptName )

Get source script path from PCART repository 获取PCART仓库中的辅助脚本路径

Parameters

scriptName The script file name under Script/

Returns: Absolute script path

Definition at line 1069 of file preprocess.py.

Here is the caller graph for this function:

◆ writeRecordValue()

def preprocess.writeRecordValue	(	filePath,
		pklRelPath,
		useCallsiteName
	)

Write recordValue.py used by instrumented project files 写入插桩文件共享的recordValue.py.

Parameters

filePath	The path of recordValue.py
pklRelPath	The relative path from recordValue.py to Copy/pkl
useCallsiteName	Whether to save pkl with callsite filename

Definition at line 1053 of file preprocess.py.

Here is the caller graph for this function:

Functions

Detailed Description

Function Documentation

◆ addDictAll()

◆ addDictSingle()

◆ codeProcess()

◆ convertLocalVar()

◆ convertTabsToSpaces()

◆ countBracket()

◆ countSpace()

◆ expandConditionalReturn()

◆ extractDecorator()

◆ findAssignCall()

◆ getAssignReceiverExpr()

◆ getDecoratorInsertIndex()

◆ getDictVar()

◆ getImportLine()

◆ getLibImportLst()

◆ getListVar()

◆ getSelfReceiverExpr()

◆ handleRunFile()

◆ ignore_sym_links()

◆ modifyFromImport()

◆ obtainDef()

◆ oneLine()

◆ saveConstantAssign()

◆ saveStructure()

◆ scriptPath()

◆ writeRecordValue()