• PHP Video Tutorials

PHP - Tokenizer Functions



Tokenizer functions can provide an interface to the PHP tokenizer embedded in Zend Engine. By using these functions, we may write our own PHP source analyzing or modification tools without having to deal with a language specification at the lexical level.

Example

<?php
   if (!defined('T_ML_COMMENT')) {
      define('T_ML_COMMENT', T_COMMENT);
   } else {
      define('T_DOC_COMMENT', T_ML_COMMENT);
   }

   $source = file_get_contents('example.php');
   $tokens = token_get_all($source);

   foreach($tokens as $token) {
      if(is_string($token)) {
         //  simple 1-character token
         echo $token;
      } else {
         //  token array
         list($id, $text) = $token;
         switch ($id) { 
            case T_COMMENT: 
            case T_ML_COMMENT: // we've defined this
            case T_DOC_COMMENT: // and this
            
			//  no action on comments
            break;

            default:
               //  anything else -> output "as is"
               echo $text;
               break;
         }
      }
   }
?>
Sr.No Function & Description
1

token_get_all() Function

This function can split a given source into PHP tokens.

2

token_name() Function

This function can get the symbolic name of a given PHP token.

php_function_reference.htm
Advertisements