Skip to main content

php tokenlizer与php-cs-fixer

· 2 min read

tokenlizer​

代码:

<?php
$code = '<?php echo "string1"."string2"; >';

$tokens = token_get_all($code);

foreach ($tokens as $token) {

    if (is_array($token)) {

        // 行号、标识符字面量、对应内容

        printf("%d - %s\t%s\n", $token[2], token_name($token[0]), $token[1]);

    }

}

输出内容:

1 - T_OPEN_TAG  <?php
1 - T_ECHO      echo
1 - T_WHITESPACE
1 - T_CONSTANT_ENCAPSED_STRING  "string1"
1 - T_CONSTANT_ENCAPSED_STRING  "string2"
1 - T_WHITESPACE

php-cs-fixer​

php-cs-fixer的核心函数是:token_get_all

        $tokens = \defined('TOKEN_PARSE')
            ? token_get_all($code, TOKEN_PARSE)
            : token_get_all($code);

调用的核心堆栈:

#0 E:\PHP-CS-Fixer\src\Tokenizer\Tokens.php(222): PhpCsFixer\Tokenizer\Tokens->setCode('<?php\n\n/*\n * Th...')
#1 E:\PHP-CS-Fixer\src\Runner\Runner.php(171): PhpCsFixer\Tokenizer\Tokens::fromCode('<?php\n\n/*\n * Th...')
#2 E:\PHP-CS-Fixer\src\Runner\Runner.php(132): PhpCsFixer\Runner\Runner->fixFile(Object(SplFileInfo), Object(PhpCsFixer\Linter\ProcessLintingResult))
#3 E:\PHP-CS-Fixer\src\Console\Command\FixCommand.php(219): PhpCsFixer\Runner\Runner->fix()
#4 E:\PHP-CS-Fixer\vendor\symfony\console\Command\Command.php(255): PhpCsFixer\Console\Command\FixCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#5 E:\PHP-CS-Fixer\vendor\symfony\console\Application.php(982): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#6 E:\PHP-CS-Fixer\vendor\symfony\console\Application.php(255): Symfony\Component\Console\Application->doRunCommand(Object(PhpCsFixer\Console\Command\FixCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#7 E:\PHP-CS-Fixer\src\Console\Application.php(84): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#8 E:\PHP-CS-Fixer\vendor\symfony\console\Application.php(148): PhpCsFixer\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#9 E:\PHP-CS-Fixer\php-cs-fixer(101): Symfony\Component\Console\Application->run()
#10 {main}

举个例子下面的堆栈:

#0 E:\PHP-CS-Fixer\src\Fixer\Operator\BinaryOperatorSpacesFixer.php(339): PhpCsFixer\Fixer\Operator\BinaryOperatorSpacesFixer->fixWhiteSpaceAroundOperatorToSingleSpace(Object(PhpCsFixer\Tokenizer\Tokens), 19)
#1 E:\PHP-CS-Fixer\src\Fixer\Operator\BinaryOperatorSpacesFixer.php(256): PhpCsFixer\Fixer\Operator\BinaryOperatorSpacesFixer->fixWhiteSpaceAroundOperator(Object(PhpCsFixer\Tokenizer\Tokens), 19)
#2 E:\PHP-CS-Fixer\src\AbstractFixer.php(75): PhpCsFixer\Fixer\Operator\BinaryOperatorSpacesFixer->applyFix(Object(SplFileInfo), Object(PhpCsFixer\Tokenizer\Tokens))
#3 E:\PHP-CS-Fixer\src\Runner\Runner.php(192): PhpCsFixer\AbstractFixer->fix(Object(SplFileInfo), Object(PhpCsFixer\Tokenizer\Tokens))
#4 E:\PHP-CS-Fixer\src\Runner\Runner.php(132): PhpCsFixer\Runner\Runner->fixFile(Object(SplFileInfo), Object(PhpCsFixer\Linter\ProcessLintingResult))
#5 E:\PHP-CS-Fixer\src\Console\Command\FixCommand.php(219): PhpCsFixer\Runner\Runner->fix()
#6 E:\PHP-CS-Fixer\vendor\symfony\console\Command\Command.php(255): PhpCsFixer\Console\Command\FixCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#7 E:\PHP-CS-Fixer\vendor\symfony\console\Application.php(982): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#8 E:\PHP-CS-Fixer\vendor\symfony\console\Application.php(255): Symfony\Component\Console\Application->doRunCommand(Object(PhpCsFixer\Console\Command\FixCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#9 E:\PHP-CS-Fixer\src\Console\Application.php(84): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#10 E:\PHP-CS-Fixer\vendor\symfony\console\Application.php(148): PhpCsFixer\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#11 E:\PHP-CS-Fixer\php-cs-fixer(101): Symfony\Component\Console\Application->run()
#12 {main}

核心就是给后面加入token // todo