SlideShare a Scribd company logo
1 of 120
Writing and using php streams and sockets
Writing and using php streams and sockets
   Idea originating in 1950’s
                   Standard way to get Input and Output
                    A source or sink of data
Definitions
                




                 C – stdin, stderr, stdout
                 C++ iostream
Who uses them




                 Perl IO
                 Python io
                 Java
                 C#
   We will write code (in pairs)
   There will be a quiz
   You get out what you put in –
    participate!

 This is really 3 talks in one -
 Streams
 Filters
 Sockets
These are the people you can blame for the PHP streams implementation
Abstracting I/O
 Access input and output generically
 Can write and read linearly
 May or may not be seekable
 Comes in chunks of data
 EVERYTHING
 include/require _once
 stream functions
 file system functions
 many other extensions
Stream
         Contexts

             Stream    Stream
ALL IO
              Filter   Wrapper
   Stream
    › Resource that exhibits a flow or succession of
     data
   Wrapper
    › Tells a stream how to handle specific
     protocols and encodings
   Context
    › A set of parameters and options to tell a
     stream (or filter) how to behave
   Scheme
    › The name of the wrapper to be used.
      file, http, https, ftp, etc.
   Target
    › Depends on the wrapper, filesystem uses a string path
      name, ssh2 uses a PHP resource

    home/bar/foo.txt
    file:///home/bar/foo.txt
    http://www.example.com/foo.txt
    ftp://user:pass@ftp.example.com/foo.txt

    php://filter/read=string.toupper|string.rot13/resource
    =http://www.example.com
Writing and using php streams and sockets
 flock
 wrapper limitations
 non-existent pointers (infinite loops can
  and will happen)
 error handling
 Parameters
 Options
 Modify or enhance a stream


 stream_context_set_param
 stream_context_set_option
 stream_context_create
Writing and using php streams and sockets
 stream_set_read_buffer
 stream_context_get_params
 stream_resolve_include_path
 stream_supports_lock
 Open_basedir
 allow_url_fopen
 allow_url_include
1.   Get a Feed from a webservice (I’m
     using flickr) -
     http://www.flickr.com/services/feeds/

2. No curl installed, allow_url_fopen is on
3. Display however you would like
Grab the feed as xml, pop it through
 simplexml, loop and display the output
 to a webpage
Streams available by default
 file://
 http://
 ftp://
 data://
 glob://
   SSL                     Phar
    ›   https://             › phar://
    ›   ftps://             Zlib
    ›   ssl://               › compress.zlib://
    ›   tls://               › zlib://
   SSH                     Bzip
    ›   ssh2.shell://        › compress.bz2://
    ›   ssh2.exec://
    ›   ssh2.tunnel://
    ›   ssh2.sftp://
    ›   ssh2.scp://
Writing and using php streams and sockets
Writing and using php streams and sockets
Writing and using php streams and sockets
 Pipes
 STDIN, STDOUT, STDERR
 proc_open
 popen
 php://stdin
 php://stdout
 php://stderr
 php://output
 php://input
 php://fd
 php://filter (5.0.0)
 php://memory (5.1.0)
 php://temp (5.1.0)
Writing and using php streams and sockets
1.   Talk to a command line program on
     your system (I’m using hg)
2.   Make it do something and read the
     output into a variable
Talk to hg, get the data, make an array of
  branches
Userland Streams
 There are no interfaces
 Implement as though there were an
  interface
 Seekable is optional
 Flushable is optional
 Directory support is optional
 and there’s even more available
 stream_open
 stream_read
 stream_write
 stream_eof
 stream_close
 stream_stat
   fopen
                 file_get_contents

Information


               Return true or false
               $this->context will have any context
                metadata
Code
   fread
                 fgets
                 file_get_contents
Information
                 etc…


                 Return string data or false
                 $this->context will have any context
                  metadata
Code
   fwrite
                 file_put_contents

Information


               get in a string of data to deal with
               return how many bytes you wrote
Code
   feof
                 file_get_contents
                 fread
Information
                 etc…


                 Return true or false
                 $this->context will have any context
                  metadata
Code
   fclose
                 file_get_contents

Information


               Don’t return anything
               any cleanup should go here
Code
   fstat
                 file_get_contents
                  file_put_contents
Information
              
Code
 fstat calls stream_stat and should always
  be implemented
 EVERYTHING ELSE uses url_stat
 Good idea to do both


   Return an array of data identical to stat()
   Called for just about everything
                 fileperms() fileinode() filesize()
                  fileowner() filegroup() fileatime()
Information       filemtime() filectime() filetype()
                  is_writable() is_readable()
                  is_executable() is_file() is_dir() is_link()
Code
 stream_seek
 stream_tell
   fseek()
                 isn’t always called – php streams have
                  read buffering on by default
Information
Code              (stream_set_read_buffer() in 5.3.3 +)
   fseek()
                 This one is ALWAYS called to get the
                  current position, even if stream_seek is
Information
Code              NOT
 stream_flush
 rename
 unlink
 stream_lock
   fflush()
                 return true if data was stored, OR there
                  was no data to store, otherwise return
Information
Code              false
   called on rename
                 no real caveats, works as described

Information
Code
   called on unlink
                 works just as described

Information
Code
   flock()
                 file_put_contents() LOCK_EX
                  stream_set_blocking()
Information
              
                 on stream close
Code
 mkdir
 rmdir
 dir_closedir
 dir_opendir
 dir_readdir
 dir_rewinddir
   mkdir()
                 watch for the recursive flag in the options
                  bitmask
Information
Code
   rmdir()
                 watch for the recursive flag in the options
                  bitmask
Information
Code
   opendir()
                  option is “enforce safe mode” (which you
                   should just ignore)
In formation
Code
   closedir()
                 all resource locked, allocated, created
                  should be released
Information
Code
   readdir()
                 should return a string filename or false
                  return value is cast to a string!
Information
              
Code
   rewinddir()
                 should reset the output generated by
                  readdir, next call should retrieve the first
Information
Code              entry
 stream_set_option() 5.3
 stream_cast() 5.3
 stream_metadata() 5.4
    › chown()
    › chmod()
    › chgrp()
    › touch()
Create a custom wrapper for any kind of
  network or shared memory storage you
  want (except s3 which is way overdone)

Implement at least the basics – extra points
  for more features
Wrapper for wincache (btw, porting this to
 apc would be a breeze)

Does basics + seek + flush (since we’re
 caching) + rename + unlink

No directories
Use Case land – when streams
make sense
 Data in s3
 Data locally during development
 Easy switch out if alternative storage is
  ever desired
 Storing image files
 Existing Zend Framework Code
 Register the s3:// wrapper
 Use a configuration setting for the stream
  to use for all images on the system
 Store and edit template files in a
  database
 Have the snappiness of including from
  disk
 Minimal Configuration
 db:// stream
 simple stream wrapper that looks for the
  template in the db, and writes it to the
  filesystem before returning the data
 The cached location is FIRST in the
  include path, so if it fails, the db stream
  gets hit
 Talk to mercurial (hg binary)
 Talk to git
 communicates via command line
 Use pipes to get data out into format for
  use in system
 Use proc_open to keep a pipe to the
  binary going
 Pass commands through stdin pipe as
  necessary
 Abstract this out to other binaries that
  are used by the system
   Test filesystem related functionality in
    simpletest or phpunit without actually
    touching the filesystem
   A stream wrapper that mocks all
    wrappable file system functionality that
    can be used with any testing system
1. Name 3 Built in PHP streams.
2. What is a Context? A Wrapper? A
   Stream?
3. How do you identify a stream?
4. Name two extensions that provide
   additional PHP streams.
 http://php.net/streams
 http://php.net/filesystem
 http://ciaranmcnulty.com/blog/2009/04/
  simplifying-file-operations-using-php-
  stream-wrappers
Because everyone needs to rot13
a file they open on the fly
 Performs operations on stream data
 Can be prepended or appended (even
  on the fly)
 Can be attached to read or write
 When a filter is added for read and
  write, two instances of the filter are
  created.
Writing and using php streams and sockets
 Data has an input and output state
 When reading in chunks, you may need
  to cache in between reads to make
  filters useful
 Use the right tool for the job
   string filters
    › string.rot13
    › string.toupper
    › string.tolower
    › string.strip_tags
   convert filters
    › convert.*
       base64-encode
       base64-decode
       quoted-printable-encode
       quoted-printable-decode
   dechunk
    › decode remote HTTP chunked encoding
      streams
   consumed
    › eats data (that’s all it does)
 bzip.compress and bzip.compress
 convert.iconv.*
 zlib.inflate and zlib.deflate
 mcrypt.* and mdecrypt.*
 Given a list of bad words (we’ll use the 7
  bad words), write a filter to remove them
  from text
 Given a commonly misspelled word, fix
  the spelling in the text
 I only did the bad words one, and
  created an extremely naïve regex that
  stripped the “bad” words from whatever
  text it finds.
 It buffers the entirety of the text before
  doing the replacement – this could be
  done by looking for word boundaries
  and doing it piecemeal to improve
  performance
Manipulate data on the fly
 Extend an internal class php_user_filter
 It’s not abstract…
 Yes that’s a horrible name
 Remember this pre-dates php 5.0
  decisions
 Note the method names are
  camelcased…
   onCreate
                 basically a constructor
                  Called every time PHP needs a new filter
Information
              
                  (on every stream)

                 return true or false
Code
   php_user_filter
    › $this->filtername
    › $this->params
    › $this->stream
   onClose
                 basically a destructor

Information




                 no return
Code
   MUST return
                  › PSFS_PASS_ON
                  › PSFS_FEED_ME
Information       › PSFS_ERR_FATAL
                 You get buckets of data and do stuff to
                  them
Code
Writing and using php streams and sockets
 $in and $out are “bucket brigades”
  containing opaque “buckets” of data
 You can only touch buckets and
  brigades with the stream_bucket_*
  functions
 You get a bucket using
  stream_bucket_make_writeable
 Given a list of bad words (we’ll use the 7
  bad words), write a filter to remove them
  from text
 Given a commonly misspelled word, fix
  the spelling in the text
 I only did the bad words one, and
  created an extremely naïve regex that
  stripped the “bad” words from whatever
  text it finds.
 It buffers the entirety of the text before
  doing the replacement – this could be
  done by looking for word boundaries
  and doing it piecemeal to improve
  performance
Use Case land – when filters
make sense
 Need to encrypt files uploaded by a user
  to add an extra security layer
 Encryption needs to happen before it
  goes “on the wire”
 Don’t want to have encryption for all files
  – don’t need overhead of adapter
  system
 Use mcrypt’s stream filtering
 Attach the mcrypt filter to the write
  stream when uploading data
 Attach the mcrypt filter to the read
  stream when downloading the data
   Upload documents to remote storage
    taking least amount of space possible in
    transmission
   Use PHP’s compressions filters (bz2, zlib) in
    conjunction with php’s streams that
    allow to compression to crush every last
    bit of space out of the files
1. What term is used to describe data
   handling in filters?
2. Name two built in filters you should be
   using and why.
3. What is the default mode filters are
   appended with?
4. What is the difference between
   appending and prepending a filter?
 http://php.net/streams
 http://php.net/filters
Putter about the network with me
   Socket
    › Bidirectional network stream that speaks a
      protocol
   Transport
    › Tells a network stream how to communicate
   Wrapper
    › Tells a stream how to handle specific
      protocols and encodings
 Network Stream, Network
  Transport, Socket Transport
 Slightly different behavior from a file
  stream
 Bi-directional data
   Sockets block
    › stream_set_blocking
    › stream_set_timeout
    › stream_select
 feof means “connection_closed”?
 huge reads or writes (think 8K)
 stream_get_meta_data is READ ONLY
 New APIS in streams and filesystem
  functions are replacements
 Extension is old and not always kept up
  to date (bit rot)
 Extension is very low level


 stream_socket_server
 stream_socket_client
Writing and using php streams and sockets
 tcp
 udp
 unix
 udg
 SSL extension
    ›   ssl
    ›   sslv2
    ›   sslv3
    ›   tls
 Given two files of quotes, write a server
  with PHP streams methods to return
  random quotes, optionally allow client to
  pick which file to retrieve quotes from
 Given a simple tcp server that will return
  a random quote – make a client that
  pulls quotes for users
 socket_server.php – uses
  stream_socket_server and
  stream_accept
 socket_client.php – uses
  stream_socket_client and a bit of loop
  magic to make a little “cli” app
Use Case land – when sockets
make sense
   Send email using an external smtp server
    with special requirements for both
    authentication and encryption
   Simple smtp class, using PHP sockets to
    communicate with the server and pass
    the mail on, complete with tls encryption
   Push data from a PHP script into flash
    player with any additional extensions or
    tools
 http://ria.dzone.com/articles/php-and-
  flex-sockets
 Use sockets to talk directly to the flash
  player
1. How is a socket different from a stream?
2. What transports does PHP provide by
   default?
3. Name the two most commonly used
   stream socket functions.
4. How can a user add additional
   transports to PHP?
 http://php.net/streams
 http://php.net/transports
 http://wikipedia.org/wiki/Unix_domain_s
  ocket
 http://tools.ietf.org/html/rfc1122
Does your brain feel like mush yet?
   Stream
              › Resource that exhibits a flow or succession of
                data
             Wrapper
Streams
              › Tells a stream how to handle specific protocols
                and encodings
             Context
              › A set of parameters and options to tell a stream
                (or filter) how to behave

             Performs operations on stream data
             Can be prepended or appended (even
              on the fly)
Filters




             Can be attached to read or write
   pipes are like sockets but only go one
                        way

Processes (pipes)
                       stdin, stdout, stderr are all pipes
                       proc_open and popen do pipes
                       You can use stream functions on
                        pipes, can’t do custom pipes


                       Socket
                        › Bidirectional network stream that speaks a
                          protocol
                       Transport
Sockets




                        › Tells a network stream how to communicate
                       Wrapper
                        › Tells a stream how to handle specific protocols
                          and encodings
 http://edit.php.net
 Blog posts
 Articles
 Cool apps using streams, sockets and
  filters properly
 Set of interfaces for streams
 Improved based filter class
 Wrappers for chmod and touch
 More tests!
 Any you have?
 http://emsmith.net
 http://joind.in/3747
 auroraeosrose@gmail.com
 IRC – freenode – auroraeosrose
 #php-gtk #coapp and others

More Related Content

What's hot

Soa cap2 exercicios resolvidos shell
Soa cap2 exercicios resolvidos shellSoa cap2 exercicios resolvidos shell
Soa cap2 exercicios resolvidos shellportal_Do_estudante
 
Swift Programming Language
Swift Programming LanguageSwift Programming Language
Swift Programming LanguageAnıl Sözeri
 
Practical Malware Analysis: Ch 8: Debugging
Practical Malware Analysis: Ch 8: Debugging Practical Malware Analysis: Ch 8: Debugging
Practical Malware Analysis: Ch 8: Debugging Sam Bowne
 
OpenWRT manual
OpenWRT manualOpenWRT manual
OpenWRT manualfosk
 
Practical Malware Analysis: Ch 7: Analyzing Malicious Windows Programs
Practical Malware Analysis: Ch 7: Analyzing Malicious Windows Programs Practical Malware Analysis: Ch 7: Analyzing Malicious Windows Programs
Practical Malware Analysis: Ch 7: Analyzing Malicious Windows Programs Sam Bowne
 
Honeypot Presentation - Using Honeyd
Honeypot Presentation - Using HoneydHoneypot Presentation - Using Honeyd
Honeypot Presentation - Using Honeydicanhasfay
 
All About Snort
All About SnortAll About Snort
All About Snort28pranjal
 
Swift Programming Language
Swift Programming LanguageSwift Programming Language
Swift Programming LanguageCihad Horuzoğlu
 
CNIT 127 Ch 6: The Wild World of Windows
CNIT 127 Ch 6: The Wild World of WindowsCNIT 127 Ch 6: The Wild World of Windows
CNIT 127 Ch 6: The Wild World of WindowsSam Bowne
 
Pwning mobile apps without root or jailbreak
Pwning mobile apps without root or jailbreakPwning mobile apps without root or jailbreak
Pwning mobile apps without root or jailbreakAbraham Aranguren
 
ADB(Android Debug Bridge): How it works?
ADB(Android Debug Bridge): How it works?ADB(Android Debug Bridge): How it works?
ADB(Android Debug Bridge): How it works?Tetsuyuki Kobayashi
 
Docker: uma visão geral e exemplos de uso
Docker: uma visão geral e exemplos de usoDocker: uma visão geral e exemplos de uso
Docker: uma visão geral e exemplos de usoTobias Sette
 
PHP, Under The Hood - DPC
PHP, Under The Hood - DPCPHP, Under The Hood - DPC
PHP, Under The Hood - DPCAnthony Ferrara
 

What's hot (20)

Soa cap2 exercicios resolvidos shell
Soa cap2 exercicios resolvidos shellSoa cap2 exercicios resolvidos shell
Soa cap2 exercicios resolvidos shell
 
Android - ADB
Android - ADBAndroid - ADB
Android - ADB
 
Swift Programming Language
Swift Programming LanguageSwift Programming Language
Swift Programming Language
 
Practical Malware Analysis: Ch 8: Debugging
Practical Malware Analysis: Ch 8: Debugging Practical Malware Analysis: Ch 8: Debugging
Practical Malware Analysis: Ch 8: Debugging
 
OpenWRT manual
OpenWRT manualOpenWRT manual
OpenWRT manual
 
Practical Malware Analysis: Ch 7: Analyzing Malicious Windows Programs
Practical Malware Analysis: Ch 7: Analyzing Malicious Windows Programs Practical Malware Analysis: Ch 7: Analyzing Malicious Windows Programs
Practical Malware Analysis: Ch 7: Analyzing Malicious Windows Programs
 
Honeypot Presentation - Using Honeyd
Honeypot Presentation - Using HoneydHoneypot Presentation - Using Honeyd
Honeypot Presentation - Using Honeyd
 
Algoritmos com java script
Algoritmos com java scriptAlgoritmos com java script
Algoritmos com java script
 
All About Snort
All About SnortAll About Snort
All About Snort
 
Swift Introduction
Swift IntroductionSwift Introduction
Swift Introduction
 
Swift Programming Language
Swift Programming LanguageSwift Programming Language
Swift Programming Language
 
Simics - Break the Rules of Product Development
Simics - Break the Rules of Product DevelopmentSimics - Break the Rules of Product Development
Simics - Break the Rules of Product Development
 
Linux systems - Getting started with setting up and embedded platform
Linux systems - Getting started with setting up and embedded platformLinux systems - Getting started with setting up and embedded platform
Linux systems - Getting started with setting up and embedded platform
 
CNIT 127 Ch 6: The Wild World of Windows
CNIT 127 Ch 6: The Wild World of WindowsCNIT 127 Ch 6: The Wild World of Windows
CNIT 127 Ch 6: The Wild World of Windows
 
Android Things : Building Embedded Devices
Android Things : Building Embedded DevicesAndroid Things : Building Embedded Devices
Android Things : Building Embedded Devices
 
Pwning mobile apps without root or jailbreak
Pwning mobile apps without root or jailbreakPwning mobile apps without root or jailbreak
Pwning mobile apps without root or jailbreak
 
ADB(Android Debug Bridge): How it works?
ADB(Android Debug Bridge): How it works?ADB(Android Debug Bridge): How it works?
ADB(Android Debug Bridge): How it works?
 
Docker: uma visão geral e exemplos de uso
Docker: uma visão geral e exemplos de usoDocker: uma visão geral e exemplos de uso
Docker: uma visão geral e exemplos de uso
 
Radius
RadiusRadius
Radius
 
PHP, Under The Hood - DPC
PHP, Under The Hood - DPCPHP, Under The Hood - DPC
PHP, Under The Hood - DPC
 

Viewers also liked

Asynchronous PHP and Real-time Messaging
Asynchronous PHP and Real-time MessagingAsynchronous PHP and Real-time Messaging
Asynchronous PHP and Real-time MessagingSteve Rhoades
 
The promise of asynchronous PHP
The promise of asynchronous PHPThe promise of asynchronous PHP
The promise of asynchronous PHPWim Godden
 
Taming the resource tiger
Taming the resource tigerTaming the resource tiger
Taming the resource tigerElizabeth Smith
 
Darkmira Tour PHP 2016 - Automatizando Tarefas com Phing
Darkmira Tour PHP 2016 - Automatizando Tarefas com PhingDarkmira Tour PHP 2016 - Automatizando Tarefas com Phing
Darkmira Tour PHP 2016 - Automatizando Tarefas com PhingMatheus Marabesi
 
Good Parts of PHP and the UNIX Philosophy
Good Parts of PHP and the UNIX PhilosophyGood Parts of PHP and the UNIX Philosophy
Good Parts of PHP and the UNIX PhilosophyYuya Takeyama
 
scdevsumit 2016 - Become a jedi with php streams
scdevsumit 2016 - Become a jedi with php streamsscdevsumit 2016 - Become a jedi with php streams
scdevsumit 2016 - Become a jedi with php streamsMatheus Marabesi
 
Writing php extensions in golang
Writing php extensions in golangWriting php extensions in golang
Writing php extensions in golangdo_aki
 
5 Tornados Intro
5 Tornados Intro5 Tornados Intro
5 Tornados IntroEcumene
 
Asynchronous I/O in PHP
Asynchronous I/O in PHPAsynchronous I/O in PHP
Asynchronous I/O in PHPThomas Weinert
 
Realtime Communication Techniques with PHP
Realtime Communication Techniques with PHPRealtime Communication Techniques with PHP
Realtime Communication Techniques with PHPWaterSpout
 
Asynchronous processing with PHP and Symfony2. Do it simple
Asynchronous processing with PHP and Symfony2. Do it simpleAsynchronous processing with PHP and Symfony2. Do it simple
Asynchronous processing with PHP and Symfony2. Do it simpleKirill Chebunin
 
TORNADOS
TORNADOSTORNADOS
TORNADOSsimran
 
Water slideshare
Water slideshareWater slideshare
Water slideshareWendy Ely
 
Angular workflow with gulp.js
Angular workflow with gulp.jsAngular workflow with gulp.js
Angular workflow with gulp.jsCihad Horuzoğlu
 
Hybrid Apps with Angular & Ionic Framework
Hybrid Apps with Angular & Ionic FrameworkHybrid Apps with Angular & Ionic Framework
Hybrid Apps with Angular & Ionic FrameworkCihad Horuzoğlu
 

Viewers also liked (20)

Asynchronous PHP and Real-time Messaging
Asynchronous PHP and Real-time MessagingAsynchronous PHP and Real-time Messaging
Asynchronous PHP and Real-time Messaging
 
The promise of asynchronous PHP
The promise of asynchronous PHPThe promise of asynchronous PHP
The promise of asynchronous PHP
 
Taming the resource tiger
Taming the resource tigerTaming the resource tiger
Taming the resource tiger
 
Php resque
Php resquePhp resque
Php resque
 
Darkmira Tour PHP 2016 - Automatizando Tarefas com Phing
Darkmira Tour PHP 2016 - Automatizando Tarefas com PhingDarkmira Tour PHP 2016 - Automatizando Tarefas com Phing
Darkmira Tour PHP 2016 - Automatizando Tarefas com Phing
 
Good Parts of PHP and the UNIX Philosophy
Good Parts of PHP and the UNIX PhilosophyGood Parts of PHP and the UNIX Philosophy
Good Parts of PHP and the UNIX Philosophy
 
scdevsumit 2016 - Become a jedi with php streams
scdevsumit 2016 - Become a jedi with php streamsscdevsumit 2016 - Become a jedi with php streams
scdevsumit 2016 - Become a jedi with php streams
 
WebSockets with PHP: Mission impossible
WebSockets with PHP: Mission impossibleWebSockets with PHP: Mission impossible
WebSockets with PHP: Mission impossible
 
Analysis edition for olap
Analysis edition for olapAnalysis edition for olap
Analysis edition for olap
 
Writing php extensions in golang
Writing php extensions in golangWriting php extensions in golang
Writing php extensions in golang
 
5 Tornados Intro
5 Tornados Intro5 Tornados Intro
5 Tornados Intro
 
Waterspout
WaterspoutWaterspout
Waterspout
 
Asynchronous I/O in PHP
Asynchronous I/O in PHPAsynchronous I/O in PHP
Asynchronous I/O in PHP
 
Realtime Communication Techniques with PHP
Realtime Communication Techniques with PHPRealtime Communication Techniques with PHP
Realtime Communication Techniques with PHP
 
Asynchronous processing with PHP and Symfony2. Do it simple
Asynchronous processing with PHP and Symfony2. Do it simpleAsynchronous processing with PHP and Symfony2. Do it simple
Asynchronous processing with PHP and Symfony2. Do it simple
 
TORNADOS
TORNADOSTORNADOS
TORNADOS
 
Water slideshare
Water slideshareWater slideshare
Water slideshare
 
RESTful API Design, Second Edition
RESTful API Design, Second EditionRESTful API Design, Second Edition
RESTful API Design, Second Edition
 
Angular workflow with gulp.js
Angular workflow with gulp.jsAngular workflow with gulp.js
Angular workflow with gulp.js
 
Hybrid Apps with Angular & Ionic Framework
Hybrid Apps with Angular & Ionic FrameworkHybrid Apps with Angular & Ionic Framework
Hybrid Apps with Angular & Ionic Framework
 

Similar to Writing and using php streams and sockets

Writing and using php streams and sockets tek11
Writing and using php streams and sockets   tek11Writing and using php streams and sockets   tek11
Writing and using php streams and sockets tek11Elizabeth Smith
 
Streams, sockets and filters oh my!
Streams, sockets and filters oh my!Streams, sockets and filters oh my!
Streams, sockets and filters oh my!Elizabeth Smith
 
Tips
TipsTips
Tipsmclee
 
Most Popular Hadoop Interview Questions and Answers
Most Popular Hadoop Interview Questions and AnswersMost Popular Hadoop Interview Questions and Answers
Most Popular Hadoop Interview Questions and AnswersSprintzeal
 
Github.com anton terekhov-orientdb-php
Github.com anton terekhov-orientdb-phpGithub.com anton terekhov-orientdb-php
Github.com anton terekhov-orientdb-phpSan jay
 
Memory allocation in c
Memory allocation in cMemory allocation in c
Memory allocation in cPrabhu Govind
 
Working with the IFS on System i
Working with the IFS on System iWorking with the IFS on System i
Working with the IFS on System iChuck Walker
 
File handling in C++
File handling in C++File handling in C++
File handling in C++Hitesh Kumar
 
Filesystem Abstraction with Flysystem
Filesystem Abstraction with FlysystemFilesystem Abstraction with Flysystem
Filesystem Abstraction with FlysystemFrank de Jonge
 
Concept of file handling in c
Concept of file handling in cConcept of file handling in c
Concept of file handling in cMugdhaSharma11
 
Basics of file handling
Basics of file handlingBasics of file handling
Basics of file handlingpinkpreet_kaur
 

Similar to Writing and using php streams and sockets (20)

Writing and using php streams and sockets tek11
Writing and using php streams and sockets   tek11Writing and using php streams and sockets   tek11
Writing and using php streams and sockets tek11
 
Streams, sockets and filters oh my!
Streams, sockets and filters oh my!Streams, sockets and filters oh my!
Streams, sockets and filters oh my!
 
File management
File managementFile management
File management
 
Tips
TipsTips
Tips
 
Files nts
Files ntsFiles nts
Files nts
 
File Organization
File OrganizationFile Organization
File Organization
 
Linux filesystemhierarchy
Linux filesystemhierarchyLinux filesystemhierarchy
Linux filesystemhierarchy
 
Most Popular Hadoop Interview Questions and Answers
Most Popular Hadoop Interview Questions and AnswersMost Popular Hadoop Interview Questions and Answers
Most Popular Hadoop Interview Questions and Answers
 
Linux
LinuxLinux
Linux
 
Github.com anton terekhov-orientdb-php
Github.com anton terekhov-orientdb-phpGithub.com anton terekhov-orientdb-php
Github.com anton terekhov-orientdb-php
 
Memory allocation in c
Memory allocation in cMemory allocation in c
Memory allocation in c
 
Unix-module3.pptx
Unix-module3.pptxUnix-module3.pptx
Unix-module3.pptx
 
Working with the IFS on System i
Working with the IFS on System iWorking with the IFS on System i
Working with the IFS on System i
 
File handling in C++
File handling in C++File handling in C++
File handling in C++
 
Download It
Download ItDownload It
Download It
 
PHP 5.3
PHP 5.3PHP 5.3
PHP 5.3
 
Filesystem Abstraction with Flysystem
Filesystem Abstraction with FlysystemFilesystem Abstraction with Flysystem
Filesystem Abstraction with Flysystem
 
Unit5
Unit5Unit5
Unit5
 
Concept of file handling in c
Concept of file handling in cConcept of file handling in c
Concept of file handling in c
 
Basics of file handling
Basics of file handlingBasics of file handling
Basics of file handling
 

More from Elizabeth Smith

More from Elizabeth Smith (20)

Welcome to the internet
Welcome to the internetWelcome to the internet
Welcome to the internet
 
Database theory and modeling
Database theory and modelingDatabase theory and modeling
Database theory and modeling
 
Taming the resource tiger
Taming the resource tigerTaming the resource tiger
Taming the resource tiger
 
Modern sql
Modern sqlModern sql
Modern sql
 
Php extensions
Php extensionsPhp extensions
Php extensions
 
Php internal architecture
Php internal architecturePhp internal architecture
Php internal architecture
 
Taming the tiger - pnwphp
Taming the tiger - pnwphpTaming the tiger - pnwphp
Taming the tiger - pnwphp
 
Php extensions
Php extensionsPhp extensions
Php extensions
 
Php extensions
Php extensionsPhp extensions
Php extensions
 
Php’s guts
Php’s gutsPhp’s guts
Php’s guts
 
Lexing and parsing
Lexing and parsingLexing and parsing
Lexing and parsing
 
Hacking with hhvm
Hacking with hhvmHacking with hhvm
Hacking with hhvm
 
Security is not a feature
Security is not a featureSecurity is not a feature
Security is not a feature
 
Using unicode with php
Using unicode with phpUsing unicode with php
Using unicode with php
 
Mentoring developers-php benelux-2014
Mentoring developers-php benelux-2014Mentoring developers-php benelux-2014
Mentoring developers-php benelux-2014
 
Using unicode with php
Using unicode with phpUsing unicode with php
Using unicode with php
 
Mentoring developers
Mentoring developersMentoring developers
Mentoring developers
 
Do the mentor thing
Do the mentor thingDo the mentor thing
Do the mentor thing
 
Spl in the wild - zendcon2012
Spl in the wild - zendcon2012Spl in the wild - zendcon2012
Spl in the wild - zendcon2012
 
Mentoring developers - Zendcon 2012
Mentoring developers - Zendcon 2012Mentoring developers - Zendcon 2012
Mentoring developers - Zendcon 2012
 

Recently uploaded

20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 

Recently uploaded (20)

20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 

Writing and using php streams and sockets

  • 3. Idea originating in 1950’s  Standard way to get Input and Output A source or sink of data Definitions   C – stdin, stderr, stdout  C++ iostream Who uses them  Perl IO  Python io  Java  C#
  • 4. We will write code (in pairs)  There will be a quiz  You get out what you put in – participate!  This is really 3 talks in one -  Streams  Filters  Sockets
  • 5. These are the people you can blame for the PHP streams implementation
  • 7.  Access input and output generically  Can write and read linearly  May or may not be seekable  Comes in chunks of data
  • 8.  EVERYTHING  include/require _once  stream functions  file system functions  many other extensions
  • 9. Stream Contexts Stream Stream ALL IO Filter Wrapper
  • 10. Stream › Resource that exhibits a flow or succession of data  Wrapper › Tells a stream how to handle specific protocols and encodings  Context › A set of parameters and options to tell a stream (or filter) how to behave
  • 11. Scheme › The name of the wrapper to be used. file, http, https, ftp, etc.  Target › Depends on the wrapper, filesystem uses a string path name, ssh2 uses a PHP resource home/bar/foo.txt file:///home/bar/foo.txt http://www.example.com/foo.txt ftp://user:pass@ftp.example.com/foo.txt php://filter/read=string.toupper|string.rot13/resource =http://www.example.com
  • 13.  flock  wrapper limitations  non-existent pointers (infinite loops can and will happen)  error handling
  • 14.  Parameters  Options  Modify or enhance a stream  stream_context_set_param  stream_context_set_option  stream_context_create
  • 16.  stream_set_read_buffer  stream_context_get_params  stream_resolve_include_path  stream_supports_lock
  • 18. 1. Get a Feed from a webservice (I’m using flickr) - http://www.flickr.com/services/feeds/ 2. No curl installed, allow_url_fopen is on 3. Display however you would like
  • 19. Grab the feed as xml, pop it through simplexml, loop and display the output to a webpage
  • 21.  file://  http://  ftp://  data://  glob://
  • 22. SSL  Phar › https:// › phar:// › ftps://  Zlib › ssl:// › compress.zlib:// › tls:// › zlib://  SSH  Bzip › ssh2.shell:// › compress.bz2:// › ssh2.exec:// › ssh2.tunnel:// › ssh2.sftp:// › ssh2.scp://
  • 26.  Pipes  STDIN, STDOUT, STDERR  proc_open  popen
  • 29.  php://filter (5.0.0)  php://memory (5.1.0)  php://temp (5.1.0)
  • 31. 1. Talk to a command line program on your system (I’m using hg) 2. Make it do something and read the output into a variable
  • 32. Talk to hg, get the data, make an array of branches
  • 34.  There are no interfaces  Implement as though there were an interface  Seekable is optional  Flushable is optional  Directory support is optional  and there’s even more available
  • 35.  stream_open  stream_read  stream_write  stream_eof  stream_close  stream_stat
  • 36. fopen  file_get_contents Information  Return true or false  $this->context will have any context metadata Code
  • 37. fread  fgets  file_get_contents Information  etc…  Return string data or false  $this->context will have any context metadata Code
  • 38. fwrite  file_put_contents Information  get in a string of data to deal with  return how many bytes you wrote Code
  • 39. feof  file_get_contents  fread Information  etc…  Return true or false  $this->context will have any context metadata Code
  • 40. fclose  file_get_contents Information  Don’t return anything  any cleanup should go here Code
  • 41. fstat  file_get_contents file_put_contents Information  Code
  • 42.  fstat calls stream_stat and should always be implemented  EVERYTHING ELSE uses url_stat  Good idea to do both  Return an array of data identical to stat()
  • 43. Called for just about everything  fileperms() fileinode() filesize() fileowner() filegroup() fileatime() Information filemtime() filectime() filetype() is_writable() is_readable() is_executable() is_file() is_dir() is_link() Code
  • 45. fseek()  isn’t always called – php streams have read buffering on by default Information Code (stream_set_read_buffer() in 5.3.3 +)
  • 46. fseek()  This one is ALWAYS called to get the current position, even if stream_seek is Information Code NOT
  • 47.  stream_flush  rename  unlink  stream_lock
  • 48. fflush()  return true if data was stored, OR there was no data to store, otherwise return Information Code false
  • 49. called on rename  no real caveats, works as described Information Code
  • 50. called on unlink  works just as described Information Code
  • 51. flock()  file_put_contents() LOCK_EX stream_set_blocking() Information   on stream close Code
  • 52.  mkdir  rmdir  dir_closedir  dir_opendir  dir_readdir  dir_rewinddir
  • 53. mkdir()  watch for the recursive flag in the options bitmask Information Code
  • 54. rmdir()  watch for the recursive flag in the options bitmask Information Code
  • 55. opendir()  option is “enforce safe mode” (which you should just ignore) In formation Code
  • 56. closedir()  all resource locked, allocated, created should be released Information Code
  • 57. readdir()  should return a string filename or false return value is cast to a string! Information  Code
  • 58. rewinddir()  should reset the output generated by readdir, next call should retrieve the first Information Code entry
  • 59.  stream_set_option() 5.3  stream_cast() 5.3  stream_metadata() 5.4 › chown() › chmod() › chgrp() › touch()
  • 60. Create a custom wrapper for any kind of network or shared memory storage you want (except s3 which is way overdone) Implement at least the basics – extra points for more features
  • 61. Wrapper for wincache (btw, porting this to apc would be a breeze) Does basics + seek + flush (since we’re caching) + rename + unlink No directories
  • 62. Use Case land – when streams make sense
  • 63.  Data in s3  Data locally during development  Easy switch out if alternative storage is ever desired  Storing image files
  • 64.  Existing Zend Framework Code  Register the s3:// wrapper  Use a configuration setting for the stream to use for all images on the system
  • 65.  Store and edit template files in a database  Have the snappiness of including from disk  Minimal Configuration
  • 66.  db:// stream  simple stream wrapper that looks for the template in the db, and writes it to the filesystem before returning the data  The cached location is FIRST in the include path, so if it fails, the db stream gets hit
  • 67.  Talk to mercurial (hg binary)  Talk to git  communicates via command line  Use pipes to get data out into format for use in system
  • 68.  Use proc_open to keep a pipe to the binary going  Pass commands through stdin pipe as necessary  Abstract this out to other binaries that are used by the system
  • 69. Test filesystem related functionality in simpletest or phpunit without actually touching the filesystem
  • 70. A stream wrapper that mocks all wrappable file system functionality that can be used with any testing system
  • 71. 1. Name 3 Built in PHP streams. 2. What is a Context? A Wrapper? A Stream? 3. How do you identify a stream? 4. Name two extensions that provide additional PHP streams.
  • 72.  http://php.net/streams  http://php.net/filesystem  http://ciaranmcnulty.com/blog/2009/04/ simplifying-file-operations-using-php- stream-wrappers
  • 73. Because everyone needs to rot13 a file they open on the fly
  • 74.  Performs operations on stream data  Can be prepended or appended (even on the fly)  Can be attached to read or write  When a filter is added for read and write, two instances of the filter are created.
  • 76.  Data has an input and output state  When reading in chunks, you may need to cache in between reads to make filters useful  Use the right tool for the job
  • 77. string filters › string.rot13 › string.toupper › string.tolower › string.strip_tags
  • 78. convert filters › convert.*  base64-encode  base64-decode  quoted-printable-encode  quoted-printable-decode  dechunk › decode remote HTTP chunked encoding streams  consumed › eats data (that’s all it does)
  • 79.  bzip.compress and bzip.compress  convert.iconv.*  zlib.inflate and zlib.deflate  mcrypt.* and mdecrypt.*
  • 80.  Given a list of bad words (we’ll use the 7 bad words), write a filter to remove them from text  Given a commonly misspelled word, fix the spelling in the text
  • 81.  I only did the bad words one, and created an extremely naïve regex that stripped the “bad” words from whatever text it finds.  It buffers the entirety of the text before doing the replacement – this could be done by looking for word boundaries and doing it piecemeal to improve performance
  • 83.  Extend an internal class php_user_filter  It’s not abstract…  Yes that’s a horrible name  Remember this pre-dates php 5.0 decisions  Note the method names are camelcased…
  • 84. onCreate  basically a constructor Called every time PHP needs a new filter Information  (on every stream)  return true or false Code
  • 85. php_user_filter › $this->filtername › $this->params › $this->stream
  • 86. onClose  basically a destructor Information  no return Code
  • 87. MUST return › PSFS_PASS_ON › PSFS_FEED_ME Information › PSFS_ERR_FATAL  You get buckets of data and do stuff to them Code
  • 89.  $in and $out are “bucket brigades” containing opaque “buckets” of data  You can only touch buckets and brigades with the stream_bucket_* functions  You get a bucket using stream_bucket_make_writeable
  • 90.  Given a list of bad words (we’ll use the 7 bad words), write a filter to remove them from text  Given a commonly misspelled word, fix the spelling in the text
  • 91.  I only did the bad words one, and created an extremely naïve regex that stripped the “bad” words from whatever text it finds.  It buffers the entirety of the text before doing the replacement – this could be done by looking for word boundaries and doing it piecemeal to improve performance
  • 92. Use Case land – when filters make sense
  • 93.  Need to encrypt files uploaded by a user to add an extra security layer  Encryption needs to happen before it goes “on the wire”  Don’t want to have encryption for all files – don’t need overhead of adapter system
  • 94.  Use mcrypt’s stream filtering  Attach the mcrypt filter to the write stream when uploading data  Attach the mcrypt filter to the read stream when downloading the data
  • 95. Upload documents to remote storage taking least amount of space possible in transmission
  • 96. Use PHP’s compressions filters (bz2, zlib) in conjunction with php’s streams that allow to compression to crush every last bit of space out of the files
  • 97. 1. What term is used to describe data handling in filters? 2. Name two built in filters you should be using and why. 3. What is the default mode filters are appended with? 4. What is the difference between appending and prepending a filter?
  • 99. Putter about the network with me
  • 100. Socket › Bidirectional network stream that speaks a protocol  Transport › Tells a network stream how to communicate  Wrapper › Tells a stream how to handle specific protocols and encodings
  • 101.  Network Stream, Network Transport, Socket Transport  Slightly different behavior from a file stream  Bi-directional data
  • 102. Sockets block › stream_set_blocking › stream_set_timeout › stream_select  feof means “connection_closed”?  huge reads or writes (think 8K)  stream_get_meta_data is READ ONLY
  • 103.  New APIS in streams and filesystem functions are replacements  Extension is old and not always kept up to date (bit rot)  Extension is very low level  stream_socket_server  stream_socket_client
  • 105.  tcp  udp  unix  udg  SSL extension › ssl › sslv2 › sslv3 › tls
  • 106.  Given two files of quotes, write a server with PHP streams methods to return random quotes, optionally allow client to pick which file to retrieve quotes from  Given a simple tcp server that will return a random quote – make a client that pulls quotes for users
  • 107.  socket_server.php – uses stream_socket_server and stream_accept  socket_client.php – uses stream_socket_client and a bit of loop magic to make a little “cli” app
  • 108. Use Case land – when sockets make sense
  • 109. Send email using an external smtp server with special requirements for both authentication and encryption
  • 110. Simple smtp class, using PHP sockets to communicate with the server and pass the mail on, complete with tls encryption
  • 111. Push data from a PHP script into flash player with any additional extensions or tools
  • 112.  http://ria.dzone.com/articles/php-and- flex-sockets  Use sockets to talk directly to the flash player
  • 113. 1. How is a socket different from a stream? 2. What transports does PHP provide by default? 3. Name the two most commonly used stream socket functions. 4. How can a user add additional transports to PHP?
  • 114.  http://php.net/streams  http://php.net/transports  http://wikipedia.org/wiki/Unix_domain_s ocket  http://tools.ietf.org/html/rfc1122
  • 115. Does your brain feel like mush yet?
  • 116. Stream › Resource that exhibits a flow or succession of data  Wrapper Streams › Tells a stream how to handle specific protocols and encodings  Context › A set of parameters and options to tell a stream (or filter) how to behave  Performs operations on stream data  Can be prepended or appended (even on the fly) Filters  Can be attached to read or write
  • 117. pipes are like sockets but only go one way Processes (pipes)  stdin, stdout, stderr are all pipes  proc_open and popen do pipes  You can use stream functions on pipes, can’t do custom pipes  Socket › Bidirectional network stream that speaks a protocol  Transport Sockets › Tells a network stream how to communicate  Wrapper › Tells a stream how to handle specific protocols and encodings
  • 118.  http://edit.php.net  Blog posts  Articles  Cool apps using streams, sockets and filters properly
  • 119.  Set of interfaces for streams  Improved based filter class  Wrappers for chmod and touch  More tests!  Any you have?
  • 120.  http://emsmith.net  http://joind.in/3747  auroraeosrose@gmail.com  IRC – freenode – auroraeosrose  #php-gtk #coapp and others

Editor's Notes

  1. Who am I, and why should you listen to me?Windows buildsCairo, PHP-Gtk, Win\\GuiUsing PHP since php 4 betaHow I got involved in PHP community and why you should tooHow I got interested in streams – the huge template debate – there were lots more but it was basically the argument that PHP is a templating language and the desire to have some really cool features using streams1 minuteStreams are one of the most powerful features of PHP and one of the most underused. Bring your laptop and go on an in-depth look at PHP’s streams layer. We'll examine what they are and how can you use (and abuse) them. Write your own classes to access APIs such as Flickr without any curl, use filters for manipulating data without altering an existing templating system, play with all sorts of PHP transports from FTP to SSH. Learn how to create your own stream wrappers and filters, and even build a socket server. Learn about the differences between socket connections and regular streams. And learn it in a hands on method, actually writing and running code. Please bring something with PHP 5.3 installed and runnable (CLI is good enough). This will have coding exercises!
  2. That’s the QR code for joind.in – should be able to point your phone at it and find it right awaySo what do you want to learn about today? Have you heard of streams, ever used them? Get some general feedback on what people think about them or have ever touched themSome thoughts on availablity of how tos and such, make everyone get up
  3. Quick computer science lessonOriginally done with magic numbers in fortran, C and unix standardized the way it workedOn Unix and related systems based on the C programming language, a stream is a source or sink of data, usually individual bytes or characters. Streams are an abstraction used when reading or writing files, or communicating over network sockets. The standard streams are three streams made available to all programs.Who else uses them? Most languages descended from C have the “files as streams concept” and ways to extend the IO functionality beyond merely files, this allows them to be merged all togetherGreat way to standardize the way data is grabbed and usedQuestions on who has used streams in other languages
  4. PHP streams are a very very big topic, could spend days talking about themSince this is a tutorial you’re going to have a VERY sore brain – ask questions, do not wait… I have kids and am VERY used to interruptions and tangentsYou won’t get anything out of this talk unless you participate, this is a tutorial not a lecturePlease get PHP 5.3 on your system, cli is fine, just some way to run this code you’re about to seePass around CDs with code/php for windows on themI will make you get up and move around – this is not because I hate you, this is because it’s early, your brains are toastFind a buddy for coding – there are prizes there as wellFormat of talk isBasics – codeAdvanced – codeQuizBreaks – we will break on time even if I’m in the middle of a slide, you do want to not wait in lineBreaks will be denoted by some of the most annoying – errr – fun slides you’ve ever seen9:00 - 12:00Breaks will be at 9:45 (10 minutes) 10:30 (15 minutes) 11:15 (10 minutes)Get up – meet and greet (5 minute mark for talk)
  5. I say “blame” and not “pat on the back” because while the idea is pretty sound, the implementation has some – well quirks. Also this was one of the least documented and used corners of PHP introduced in PHP 4.3 to unify the methods for working on files, sockets, and other similar resourcesThere are lots of other people who have dabbled their fingers in the streams code, and lots of cool features related to it
  6. So what’s the basic stream that everyone uses? It’s not even stdin or stdout – it’s the most basic of all file://Who has ever used fopen or include? Any of those calls are actually using the streams layer in PHPA stream is a way of generalizing file, network, data compression, and other operations which share a common set of functions andusesPHP manual states that a stream is a resource object which exhibits streamable behavior. That is, it can be read from or written to in a linear fashionSo if you have data you want to read or write in a linear fashion – streams are for you!Should be at the 5 minute mark (maybe later)
  7. Streams are a huge underlying component of PHPStreams were introduced with PHP 4.3.0 – they are old, but underuse means they can have rough edges… so TEST TESTTESTBut they are more powerful then almost anything else you can useWhy is this better ?Lots and lots of data in small chunks lets you do large volumes without maxing out memory and cpu
  8. Any good extension will use the underlying streams API to let you use any kind of streamfor example, cairo does thisstuff to work with PHP streams is spread across at least two portions of the manual, plus appendixes for the build in transports/filters/context options. It’s very poorly arranged so be sure to take the time to learn where to look in the manual – there should be three main placesWhat doesn’t use streams? Chmod, touch and some other very file specific funtionality, lazy/bad extensions, extensions with issues in the libraries they wrap around
  9. All input and output comes into PHPIt gets pushed through a streams filterThen through the streams wrapperDuring this point the stream context is available for the filter and wrapper to useStreams themselves are the “objects” coming inWrappers are the “classes” defining how to deal with the stream
  10. What is streamablebehavorior? We’ll get to that in a bitResource: A resource is a special variable, holding a reference to an external resourceThis is something PHP uses to point to an underlying C thingya wrapper tells the stream how to behave – how to “speak” http or to the file system or whateverYour context lets you set options that are made available to your wrappers and filtersIF YOU REMEMBER NOTHING ELSE FROM THIS TALK< REMEMBER THESE!
  11. How the schemes for the streams filters work. The php stream is “magic” because it lets you tack on filters for things that don’t leave a file pointer lying around (include, file_get_contents)Basically this is a version of a URI – only not quite. Because we really can’t seem to make anything truly standard in PHP
  12. Some notes – file_get_contents and it’s cousin stream_get_contents are your fastest most efficient way if you need the whole fileFile(blah) is going to be the best way to get the whole file split by linesBoth are going to stick the whole file into memory at some point.For very large files and to help with memory consumption, the use of fgets and fread will help
  13. Error handling is a major pain with the file system code in PHP, and with streams codeThere is no way to deal with “expected errors” and the code is noisy with warnings. Your best bet is to have an error handler registered to deal with warnings when using file system stuffYou can also use some of the spl file classes – such as splfileobject – to do some of the filesystem functions in an object oriented wayFlock issues are numerousEverything has to be using php’s flock – you can use locking in python with php’s flock and expect it to workYes it works on windows – but it does NOT work on threaded servers (cgi yes, mod_php no)Some versions of linux have broken flock implementationsIs not going to work on network sharesRemember a wrapper is going to define the PHP streams stuff it supports. Some don’t implement the functionality needed for things like mkdir or is_dir, some arent’ seekable
  14. A context is a set of parameters and wrapper specific options which modify or enhance the behavior of a stream. There is really only one parameter that can be set for streams and that is the notification callbackso you can assign a callback to your stream that is called when things happen – check for the notify constant coming it – the possible values are available in the manual, then do something (logging maybe) depending on what is happening with the streamoptions are really the more useful of the twoBasically they’re arrays of settings that are then made available – if you need additional data inside a custom stream wrapper contexts are going to be the way to go
  15. It’s all in the contextall the http functionality is controlled by the context settings – if you go to php.net there’s an appendix with all the context settings available for the http (and https) requestsmethod , header, user_agent,content, proxy , request_fulluri,max_redirects,protocol_version ,timeout ,ignore_errors are just a few available.This may seem VERY backward – but content goes in the context and then you do a file_get_contents, even when POSTing or putting data (which feels a bit odd but it’s the way the http stream is implemented)
  16. Normally PHP has 8K buffers in place, you could always change this with stream_set_write_buffer, this is the pair method for thatget_params is a pair to stream_context_set_paramswhich has been available since the beginning like set write bufferstream_resolve_include_path acts like `include` and will only resolve the filename against the include-path, if the path is relative. It makes no sense to resolve already absolute pathnames anyway.the last one is pretty self explanatory
  17. These three things can royally mess up stream based codeIf you’re writing a system that must run with these completely locked down you may run into some serious issuesOpen_basedir is just generally buggyEnd at 9:15 or so
  18. 9:15 to 9:20 coding exercise
  19. So let’s think of a really simple problem here that uses PHP streams9:20-9:25 for show and tell
  20. So now we’re going to take a look at all the shiny C level transport stuff available for the systemThis is stuff not only available and built in to PHP by default, but also stuff available with additional extensions. There are several extensions that provide additional built in transports, streams and filters – we’ll hit the coolest oneswarning: quite a lot of this stuff is going to be affected by allow_url_include and allow_url_fopenit’s probably better for security reasons that at least allow_url_include is turned off – but allow_url_fopen, you could really argue that either wayBottom line is when you have something so powerful you can replace a regular file include with a remote include from a url you need to be extra carefulSecurity should be your first name, not just your middle name, when dealing with streams and filters
  21. These are the native built in streams for PHP – no extension necessaryNote that the file:// is considered the default and no prefix means “file” to PHPglob is an odd duck, new since PHP 5.3data is available since 5.2 and follows the RFC 2397http and ftp are pretty self-explanatory – both have context settings that make them really useful
  22. You do need to note that there are two kinds of “transports” – those you can use with regular streams and those that are socketscontrary to popular believe the https/ftps/ssl and tls stuff is not really built into PHP by default –it’s part of the open ssl extension. However, the streams will register even when using openssl as a shared module, just make sure you’re loading it via .ini and not dl() – you could get some interesting behaviors.Ssh2 has the majority of it’s api – especially sftpapi, totally done with streams, which takes a bit of getting used to at first but it is a very powerful if not very verbose way of doing things. It does make it possible to drop in ssh2 stream paths where regular file actions used to be completely transparentlyThe compression stuff is often taken for granted as well since most PHP systems have zlib installed, and often bzip tooogg and exprect exist as well and are a little more esoteric, not many people use them so I don’t have examples for themPhar is a really really cool way to package up apps and have them “just run”
  23. https://www.x.com/docs/DOC-1632An example of how to do instant payment notifications at paypal – a lot of the script is chopped out – for example building the headers and request data, but the basics of build your data, then open your fsockopen and do your fputs to send the dataWhy do we do the craziness with feof?Because the socket can hang forever! If a connection opened by fsockopen() wasn't closed by the server, feof() will hangHanging while feof is very very badYou have to be careful in general with feofReturns TRUE if the file pointer is at EOF or an error occurs (including socket timeout); otherwise returns FALSE
  24. First we create the connection and auth – you can also use keyfiles with ssh2Then we’re going to use exec to clear out the old files first, notice the stream set blocking – we’re saying “wait until this sucks is done and you get something back”Then clean up after ourselves – there is an ssh2.exec:// stream, but it makes little sense here, we’re not going to use the output of the exec for anythingThen we do and is_dir on the remote and mkdir, and copy our files overNote the user of the actual resource in the url – that’s some really dirty magic under the covers in PHP to make that work
  25. Some things to note about phars:they aren’t affected by allow_url_fopen (they can NEVER work on remote files)requires system-level disabling of the phar.readonly php.ini setting in order to create or modify phar archivesIf you’re usingphar without an exectuable stuff (just for zip or tar archives) that limitation is not in placePhars are really quite awesome. Note that phar is on by default for 5.3openssl can be used for signingzlib and bzip can be used for compression – but both of these are optional for phar
  26. pipes are like sockets but only go one waystdin, stdout,stderr are all pipesproc_open and popen do pipesSTDIN/out/err are some really nifty magic in PHP, we’ll get to them later, but they’re easy ways to access and write to standard pipes from PHPproc_open is popen on steroidsyou can do mutiple pipes and suppress that ugly black box on windows and perform other black magicYou DO have to be careful if you are using this stuff, remember windows is going to act quite a bit differently in some cases – proc_open and popen are interesting on there at best
  27. php://stdin, php://stdout and php://stderr allow direct access to the corresponding input or output stream of the PHP process. The stream references a duplicate file descriptor, so if you open php://stdin and later close it, you close only your copy of the descriptor--the actual stream referenced by STDIN is unaffected. Note that PHP exhibited buggy behavior in this regard until PHP 5.2.1. It is recommended that you simply use the constants STDIN, STDOUT and STDERR instead of manually opening streams using these wrappers.
  28. php://output allows you to write to the output buffer mechanism in the same way as print() and echo(). php://input allows you to read raw POST data. It is a less memory intensive alternative to $HTTP_RAW_POST_DATA and does not need any special php.ini directives. php://input is not available with enctype="multipart/form-data". NOTE: you can’t attach a filter to say output and then echo and expect it to magically work, instead you must fwrite to outputphp://stdin and php://input are read-only, whereas php://stdout, php://stderr and php://output are write-only. php://input allows you to read raw POST data. It is a less memoryintensive alternative to $HTTP_RAW_POST_DATA and does not need anyspecial php.ini directives. php://input is not available withenctype="multipart/form-data"php://fd – this is new in 5.3.6. php://fd allows direct access to the given file descriptor. For example, php://fd/3 refers to file descriptor 3.
  29. php://filter is a kind of meta-wrapper designed to permit the application of filters to a stream at the time of opening. This is useful with all-in-one file functions such as readfile(), file(), and file_get_contents() where there is otherwise no opportunity to apply a filter to the stream prior the contents being read. so you’re getting a filter and sticking it on a resource – read filter chain, write filter chain, Any filter lists which are not prefixed specifically by read= or write= will be applied to both the read and write chains (as appropriate). The php://memory wrapper stores the data in the memory. php://temp behaves similarly, but uses a temporary file for storing the data when a certain memory limit is reached (the default is 2 MB). The php://temp wrapper takes the following 'parameters' as parts of its 'path': /maxmemory:<number of bytes> (optional). This parameter allows changing the default value for the memory limit (when the data is moved to a temporary file). so you can actually manage the memory limit for when the data moves to a temp file
  30. This is actually from Ciaran McNulty ‘s blog and it was such a perfect example of how PHP streams can make your life easierWhat if later you need to copy that file from ssh or it’s compressed with a different algorithm? That’s a lot of code tweaks. Where as in the second version simply changing the stream is enough to fix it9:25 – first large chunk of brain dumping done
  31. 9:35 to 9:40 coding exercise
  32. So let’s think of a really simple problem here that uses PHP streams9:45 to 9:50 show and tellTHEN - time for first break (10 minutes)
  33. Custom streams and filters are very useful tools, they let you have all kinds of power over how your application actsThrough filter and stream chaining you can link several types of streams together for some very powerful actionsUserland filters are not constrained by allow_url_fopen, but if they try to use streams internally they will failObjects for userland streams may not work the way you expect, so you’ll need to test your stuff pretty carefullyFor better or worse, stream contexts are read only inside a stream wrapper. If you try to change them very bad things can happenStart this at 10:00
  34. There should be interfaces for streamsYes I have a patchYes I am a bad bad girl for not getting that patch in yetThere are a small list of functions that you absolutely must implement in order to make a custom stream workCustom streams allow you to control the behavior of your data but there are some things you need to knowWhy it is a good idea to implement your own interfaces for streams – THERE ARE NO CHECKS ON wrapper_register. The only checks you’ll get is when the call actually takes place that you want to wrap (less than useful)
  35. These are the basics you should always implement – if you do not PHP is going to spit warningsOpen up and look at my base stream interface for a quick idea of what this would look like
  36. This is usually the heart of a custom stream wrapper. It’s important that you Return true or falseCheck options for STREAM_USE_PATH – if that’s set then you must put the actual path you used into opened_pathIf path is relative, search for the resource using the include_path is what it meansThe mode will be identical to the modes for fopen – so you’ll get a string you’ll need to check. Make sure you’re checking for bad modes! Path can generally be broken up pretty efficiently with url_parse since the general format of a PHP stream path is identical to a url ;)If you support contexts, $this->context will have the data that is being set/passed around inside your current stream context.The constructor of your class IS called every time before stream_open!
  37. You’ll generally want to be keeping track of the current position of your stream pointer in your class. Generally you do this with a position class property. For fread, you’ll want to update that with the number of bytes that are actually read, not necessarily the number requested by fread.warning! if you return an empty string PHP is going to treat it as “false” AND returning false does not necessarily indicate eof, so be careful when you implement this.However you MUST make it return false or ‘’ at some point, or it’ll infinite loop and NEVER call eof for file_get_contents for fread/fgest
  38. If you don’t’ have enough room for all the data that you’re sent, write as much as possibleIf the return value is greater the length of data, E_WARNING will be emitted and the return value will truncated to its length.
  39. Feof is kind of a magical thing. It’ll end up being called all over the place when you least expect it!. And then not when you want it to be called (ick)Notice that the constructor is NOT called before a eofThis is called with feof, file_get_contents, fread. Anything that might need to know if we’re all done with data. So this should probably be the second thing you implement in your stream wrapperJust like stream_open you need to return true or false and can access any context information, but it has no parameters passed.
  40. You need to put up your cleanup stuff here, because if you put it in a PHP destructor you can get some really really odd behaviorNote this method doesn’t give you anything (no params) and you shouldn’t return anything (well you can but it won’t do you any good)this is a bit broken though! if you don’t implement stream_close it doesn’t throw a warning unlike the other methods. It’s generally safest to always implement stream_close
  41. Although technically you might not need to implement this – if you do any file_get_contents or file_put_contents calls into your code this WILL be calledNote the data being sent back is used for other stuff – the mode stuff can get really annnoying – check out the unix man page for how your wrapper should be handling mode stuff to get it to do what you want
  42. file_get_contents is going to call this! and just about everything else under the sun – you really need to have it implemented.Notice there are slightly different semantics between the two – url_stat is going to give you a path and optionally flags to deal withSTREAM_URL_STAT_LINK For resources with the ability to link to other resource (such as an HTTP Location: forward, or a filesystemsymlink). This flag specified that only information about the link itself should be returned, not the resource pointed to by the link. This flag is set in response to calls to lstat(), is_link(), or filetype().STREAM_URL_STAT_QUIET If this flag is set, your wrapper should not raise any errors. If this flag is not set, you are responsible for reporting errors using the trigger_error() function during stating of the path.
  43. SOME NOTES: so why two stats? one works on an already opened stream, one is given only a path and needs to determine informationfrom that that means stream open will NOT have been called when this is called!
  44. Generally you need to implement both if your stream is going to have any kind of seekable behaviorNote that you need both! I’ll repeat that again, you must have both! one will tell the wrapper where you are, one will tell the wrapper to move your position in the stream
  45. Upon success, streamWrapper::stream_tell() is called directly after calling streamWrapper::stream_seek(). If streamWrapper::stream_tell() fails, the return value to the caller function will be set to FALSENot all seeks operations on the stream will result in this function being called. PHP streams have read buffering enabled by default (see also stream_set_read_buffer()) and seeking may be done by merely moving the buffer pointer.
  46. You’ll need to have some way inside your class/wrapper to keep track of where you are – stream tell is almost always simply returning the value of that position pointers – don’t try to make it any more complicated then that
  47. These are all items that are going to be very specific to the type of stream you are implementingnot every stream will need or want them, but they can be very useful
  48. flushing again – you generally only need to implement this if you’re doing some buffering or caching in your custom stream layerSo why would you need to implement flushing? This is most useful if you’re doing some kind of storage in your file, waiting for a decent size to send, waiting for more data to write, etc. This should also clear any caching you have going related to stat
  49. These are pretty self explanatory as well
  50. Implementing these is REALLY going to depend on what you’re doing and how you’re implementing it and what you want to do with your streamLocking This method is called in response to flock(), when file_put_contents() (when flags contains LOCK_EX), stream_set_blocking() and when closing the stream (LOCK_UN). - Portable advisory file locking
  51. If your wrapper does not support any directory functionality – do NOT define these in your class
  52. A bitwise mask of values, such as STREAM_MKDIR_RECURSIVE. - this is a rather undocumented constant that you need to dig out of the options
  53. public boolstreamWrapper::rmdir ( string $path , int $options )A bitwise mask of values, such as the completely undocumented STREAM_MKDIR_RECURSIVE.
  54. Specifies the URL that was passed to opendir(). Note: The URL can be broken apart with parse_url(). options Whether or not to enforce safe_mode (0x04).
  55. This method is called in response to closedir(). Any resources which were locked, or allocated, during opening and use of the directory stream should be released.
  56. Should return string representing the next filename, or FALSE if there is no next file. Note: The return value will be casted to string.
  57. This method is called in response to rewinddir(). Should reset the output generated by streamWrapper::dir_readdir(). i.e.: The next call to streamWrapper::dir_readdir() should return the first entry in the location returned by streamWrapper::dir_opendir().
  58. STREAM_OPTION_BLOCKING (The method was called in response to stream_set_blocking())STREAM_OPTION_READ_TIMEOUT (The method was called in response to stream_set_timeout())STREAM_OPTION_WRITE_BUFFER (The method was called in response to stream_set_write_buffer())Can be STREAM_CAST_FOR_SELECT when stream_select() is calling stream_cast() or STREAM_CAST_AS_STREAM when stream_cast() is called for other uses. Should return the underlying stream resource used by the wrapper, or FALSE. this “finishes out’” file functionality by allowing wrappers for basically everything that touches the file systemThis makes things like chown, chmod, chgrp and touch work properly, which is pretty cool actually because it makes
  59. So let’s think of a really simple problem here that uses PHP streams10:00 – 10:15 coding time
  60. So let’s think of a really simple problem here that uses PHP streams10:15 to 10:20 show and tell
  61. So enough of the runaround, we’ve learned what PHP streams and filters are, we’ve learned how to use them, we’ve learned how to use built in streams, filters, and transports and how to create custom streams and filters. Now lets take a look at a couple of use cases for PHP streamsA use case in software engineering and systems engineering is a description of a system’s behavior as it responds to a request that originates from outside of that system. In other words, a use case describes "who" can do "what" with the system in question. The use case technique is used to capture a system's behavioral requirements by detailing scenario-driven threads through the functional requirements.So what is a use case and why are we bothering – so many times I hear “these are awesome, but when would I ever use it?”
  62. Theoretical company wants to store lots of dataNeeds a way to transparently change where and how that data is storedHave to keep it flexible enough that when the next guy down the line changes his mind as to HOW the data is stored minimal changes will be necessary
  63. This also filters over into using an entire template system based on streams instead of really abstract object orientation – this allows the implementation to be very simple and to get complex behavior, you add in custom filters and streamstalk about zend frameworks stream wrapper and phpbb’stemplating using it as well
  64. So this is actually a clever idea I found in a piece of code by Wez Furlong - mtrack (basically a trac clone in PHP)
  65. https://github.com/mikey179/vfsStream found this while poking around the internet, interesting idearealpath() does not work with any other URLs than pure filenameschmod(), chown() and chgrp() can not be used in conjunction with vfsStream URLs due to limitations of stream wrappers, which do not support changing file modes, owners or owner groups. Update: Probably this can be fixed with PHP 5.4, which adds support for this to stream wrappers.touch() does not work with any other URLs than pure filenames. Workaround: fclose(fopen($file, 'a')); ext/zip seems not support userland stream wrappers, so it can not be used in conjunction with vfsStreamis_executable() on a vfsStream directory always returns false - this is a problem with PHP itself, see comments on the is_executable() manual page
  66. This is a long talk, so ask now! I’ll either say “I’ll talk about that in filters” or “I’ll talk about that with sockets” or answer it ;)
  67. And in comes part 2 – starting at 10:45Filters are actually incredibly powerful things
  68. A filter is a final piece of code which may perform operations on data as it is being read from or written to a stream. Any number of filters may be stacked onto a stream. Custom filters can be defined in a PHP script using stream_filter_register() or in an extension using the API Reference in Working with streams. To access the list of currently registered filters, use stream_get_filters(). Stream data is read from resources (both local and remote) in chunks, with any unconsumed data kept in internal buffers. When a new filter is prepended to a stream, data in the internal buffers, which has already been processed through other filters will not be reprocessed through the new filter at that time. This differs from the behavior of stream_filter_append(). Filters are nice for manipulating data on the fly – but remember you’ll be getting data in chunks, so your filter needs to be smart enough to handle that
  69. Filters can be appended or prepended – and attached to READ or WRITENotice that stream_filter_prepend and append are smart – if you opened with the r flag, by default it’ll attach to read, if you opened with the w flag, it will attach to writeNote: Stream data is read from resources (both local and remote) in chunks, with any unconsumed data kept in internal buffers. When a new filter is prepended to a stream, data in the internal buffers, which has already been processed through other filters will not be reprocessed through the new filter at that time. This differs from the behavior of stream_filter_append(). Note: When a filter is added for read and write, two instances of the filter are created. stream_filter_prepend() must be called twice with STREAM_FILTER_READ and STREAM_FILTER_WRITE to get both filter resources.
  70. Well it may look like manipulating data in a variable is preferable to the above. But the above is just a simple example. Once you add a filter to a stream it basically hides all the implementation details from the user. You will be unaware of the data being manipulated in a stream.And also the same filter can be used with any stream (files, urls, various protocols etc.) without any changes to the underlying code.Also multiple filters can be chained together, so that the output of one can be the input of another.The filters need an input state and an output state. And they need torespect the the fact that number of requested bytes does not necessarilymean reading the same amount of data on the other end. In fact the outputside does generally not know whether less, the same amount or more input isto be read. But this can be dealt with inside the filter. However thefilters should return the number input vs the number of output filtersalways independently. Regarding states we would be interested if reachingEOD on the input state meant reaching EOD on the output side prior to therequested amount, at the requested amount or not at all yet (more dataavailable).
  71. The string filters are… well slightly less then useful. The only really useful thing about them is you can use some clever php magic to transparently stick them on a fileReally limited in usefulness – what would make them more useful?However, if you’re writing a PHP extension and need to see how the filter stuff works under the hood grab /ext/standard/filters.c and you’ll see some great examples
  72. The convert filters are also not really usefulWhat does that star means? means there’s a whole bunch of convert options available but it’s all linked into one wildcard filterhowever, the open in the PHP source checks for these four itemsThese probably aren’t useful in most of your work, unless you’re doing some evil mail stuffThis is most useful for seeing how a filter handles a wildcard situation – you can register and deal with your own wildcard filters if you wantdechunk does exactly what it says and deals with chunked encodingThe consumed one is a bit of an odd duck – it eats the data and basically throws it away
  73. There are some extensions with some filters, and oddly enough they tend to be much more usefulon the fly compression ( a filter is actually more useful then a stream in some caseson the fly encryption, on the fly iconv conversion!11:05 – 5 minutes for “what are streams” now for “how to make streams even cooler”
  74. With bucket brigade exercise, 15 minutes to get to here11:0010 minute write a filter exercise, 5 minutes of show and tell
  75. 11:15 wrap up filter exercisesShow code, talk about solution
  76. So just like streams, you have the capability of doing your own implementation of PHP filters
  77. so for custom filters you’re going to extend this php_filter_classThis class predates any internal PHP classes, so it predates the naming conventions (and of course we NEVER EVER want to break BC) note that the methods however are camelcased not underscored, because you know everything needs to be screwy and inconsistentI really want a nice abstract class for this… So you extend the class and then use stream_register_filter to register itRemember you can “wildcard” your user stream to do multiple filters in one
  78. This method is called during instantiation of the filter class object. If your filter allocates or initializes any other resources (such as a buffer), this is the place to do it. Your implementation of this method should return FALSE on failure, or TRUE on success. When your filter is first instantiated, and yourfilter->onCreate() is called, a number of properties will be available as shown in the table below.
  79. A string containing the name the filter was instantiated with. Filters may be registered under multiple names or under wildcards. Use this property to determine which name was usedThe contents of the params parameter passed to stream_filter_append() or stream_filter_prepend()The stream resource being filtered. Maybe available only during filter calls when the closing parameter is set to FALSE.
  80. This method is called upon filter shutdown (typically, this is also during stream shutdown), and is executed after the flush method is called.
  81. PSFS_PASS_ON Filter processed successfully with data available in the outbucket brigade.PSFS_FEED_ME Filter processed successfully, however no data was available to return. More data is required from the stream or prior filter. PSFS_ERR_FATAL (default) The filter experienced an unrecoverable error and cannot continue. Note that you CANNOT change that signature – you MUST accept consumed by referenceClosing tells you if the filter
  82. Sometimes a picture is worth a thousand wordsthis is how data is handled in filters – the bucket brigade is the line of people passing those buckets full of water (or in our case data) to the fire
  83. So – you get a bucket (using stream_bucket_make_writeable)The name is a misnomer – what it is actually doing is grabbing a bucket from the brigade… it’s the handoffSo we get the handoff and then we do something to itBuckets are actually a little object with a datalen and data insideThen when we’re done with it, we append to preprend it to the out brigade (we’re done with the bucket, we filled it with water, we’re passing it off)Note that you can also create an entirely NEW bucket and whack it into brigade if you want!You’ll most definitely want to keep track of your current state in your filter class. Remember you can rip data out of the bucket, store it in an internal property … and then say “feed me” to get more data
  84. With bucket brigade exercise, 15 minutes to get to here11:0010 minute write a filter exercise, 5 minutes of show and tell
  85. 11:15 wrap up filter exercisesShow code, talk about solution
  86. Filters are probably the very least used thing in PHP, not because they aren’t useful, but because people don’t see a solution for themIn fact this is true for streams and filters, I see a LOT of people using layers and layers of Object Oriented programming and adapter patterns and decorators and intercepting filters and blah blahblahWhen all they really need is a simple filter on a stream
  87. http://jeremycook.ca/2011/I’m building an app where I need to encrypt files uploaded by a user to add an extra security layer. I was initially thinking of using stream_filter_register() to create my own stream filter as the files are read and written. If you’re not familiar with the concept of stream filters in PHP they’re a very powerful feature. By attaching a filter to a stream you can perform various operations to data as it is being read from or written to a stream. Once the filter is defined and attached to the stream this is done completely transparently. Anyway, coming back to my problem of encrypting files, I did a quick search on Google and there didn’t seem to be an easy way of doing this. I then came across this gem in the PHP manual. PHP has a number of built in stream filters and one of them is an encryption filter. Providing you have the mcrypt extension installed encrypting and decrypting files is as easy as registering a stream filter on a stream! Pasted below is the example code from the PHP manual.03/20/easy-file-encryption/
  88. 11:25 start back upBucket BrigadeAny built in filter By default, stream_filter_append() will attach the filter to the read filter chain if the file was opened for reading (i.e. File Mode: r, and/or +). The filter will also be attached to the write filter chain if the file was opened for writing (i.e. File Mode: w, a, and/or +). STREAM_FILTER_READ, STREAM_FILTER_WRITE, and/or STREAM_FILTER_ALL can also be passed to the read_write parameter to override this behavior.Order in which they’re added to the filter list AND append will always be reprocessed, prepend will not
  89. This is your last chance to get question in, here are the socket resource links
  90. Sockets and streams are “interchangeable” to a point in PHPYou can ONLY use available registered transportsTransports cannot be created at the PHP level, only at the C levelBut can be added via PHp extensions – some extensions that have transports include the ssl extension
  91. What is streamablebehavorior? We’ll get to that in a bitProtocol: set of rules which is used by computers to communicate with each other across a networkResource: A resource is a special variable, holding a reference to an external resourceTalk about resources in PHP and talk about general protocols, get a list from the audience of protocols they can name (yes http is a protocol)A socket is a special type of stream – pound this into their headsA socket is an endpoint of communication to which a name can be bound. A socket has a type and one associated process. Sockets were designed to implement the client-server model for interprocess communication where:Inphp , a wrapper ties the stream to the transport – so your http wrapper ties your PHP data to the http transport and tells it how to behave when reading and writing data
  92. Sockets are just like "worm holes" in science fiction. When things go into one end, they (should) come out of the other. According to the unix socketsfaqIn fact, if you are using Unix, sockets actually are filesAnyway – so if sockets are these worm holes and we can use them just like files with our fwrite, and fread and friends, we can have fun
  93. Some quick notes:blocking is like sleeping – it’s synchronous – nothing is going to happen until it’s doneYou can use stream_set_blocking to get around this – which means reads and writes will fail instead of blockingyou have to check the values from fread and fwrite and if they’re zero, try again (send the data again)you’ll need internal buffering and feof has no meaningthis is buggy as hell under windows – particularly with processes – works really quite well with sockets – mixed results with streams\\your mileage may varyYou can use stream_set_timeout – defaults to 60 seconds, after that sets “timed_out” in meta data and returns empty string/zerotimeouts are really only useful with one socketblocking is a PAIN to get working correctly but very useful when doing a lot of things at oncestream_select is also buggy on windows – especially with processes (the processes stuff with PHP on windows is …. icky)it does timeouts and blocking basically – tells you when what you want to do will NOT blockfeof does NOT MEAN CONNECTION CLOSEDit means either a read failed and the buffer is empty ORbuffer is empty and there is no data within the timeoutYou’re moving data across the black hole – do yourself a favor and do it in little chunks. Will make the world a better placeand while stream_get_meta_data has some awesome information don’t be poking at it, it’s for information purposes only
  94. Avoid the old sockets extension unless you really really know what you’re doingMost of the things you used to need the sockets extension for you no longer dothose last two functions, stream socket server and stream socket client make doing a client/server relationship really easy with much less codeIt’s sometimes hard to find examples on the stream_socket stuff since most of the old stuff on the internet still uses the sockets extensionDon’t follow their lead, take the time to read the php documentation and use the new APIs
  95. By default sockets are going to assume tcp – since that’s a pretty standard way of doing things. Notice that we have to do things the old fashioned way just for this simple http request – sticking our headers together, making sure stuff gets closed. However if you can’t use allow_url_fopen this is a way around ita dirty dirty way but – there you have itremember allow_url_fopen only stops “drive-by” hacking
  96. Internet Domain sockets expect a port number in addition to a target address. In the case of fsockopen() this is specified in a second parameter and therefore does not impact the formatting of transport URL. With stream_socket_client() and related functions as with traditional URLs however, the port number is specified as a suffix of the transport URL delimited by a colon. unix:// provides access to a socket stream connection in the Unix domain. udg:// provides an alternate transport to a Unix domain socket using the user datagram protocol. Unix domain sockets, unlike Internet domain sockets, do not expect a port number. In the case of fsockopen() the portno parameter should be set to 0.
  97. 11:35 to here – home stretch10 minute code exercise and 5 minute show and tellFigure out IP beforehand and make sure my solution is runningPick one or the other – server will be running on my system for testing for client creators, client can literally be two lines
  98. Wrap up at 11:50 – run rest to endFlip to demo code in komodo
  99. You shouldn’t ever learn about something in PHp without the why as well as the what and how – so while I’ve focused quite a bit on what and how I’m also trying to focus on why – use cases that make sense
  100. It requires a “transport” – a specific type of PHP stream wrapper that talks to the networkTcp, udp, unix, udgStream_socket_server and stream_socket_clientHas to be done through a C level extension
  101. This is your last chance to get question in, here are the socket resource linksThe rfc there is for the tcp/ip modelThe unix domain sockets page is pretty nice if you want to figure out exactly how the unix sockets work – one thing you do have to remember is unix sockets are not available and won’t work on windows
  102. Do some review here with questions for the audienceWhat are streams?What are sockets?What are filters?Do you have ideas for places you would now use them in your own code?Can you think of some standard libraries that should be built? What frameworks or libraries have good tools for this stuff?
  103. So how do you spread the word and help make streams sockets and filters something to be used more “in the wild”First of all, the documentation can always use a hand, visit edit.php.net for an online docbook editor to add stuff to the manualCode sampels and blog posts and articles are also a good good thingBut most of all get your own applications using it and showcasing the cool opportunities for this kind of stuff – put in an s3 stream (I dare someone to get it working properly with wordpress – oy) pick any open source project and get it fully stream compatible, the results might surprise you
  104. These are things that I want to haveinphp – the biggest thing are the interfaces for streams – even if they’re only “helpers” in 5.3 and or 5.4, to have a future where you can easily see that something is say seekable would be great. Also some heavy duty testing would be great, especially for extensions and edge casesWhat else would you like to see? I’m kind of curious
  105. Let’s wrap it up and wake up