Facebook can be seen from the GitHub account, open source projects open source Facebook has nearly 300, covering areas such movement, the front-end, Web, back-end, big data, database, tools and hardware. Facebook open source project leader James Pearce had OSCON Facebook explained exactly why you want to use, support and publish open source projects. details as follows:
Share Facebook code (usually software & ldquo; & rdquo ;, occasionally stack, including hardware design) to promote innovation in this world. These codes help others develop software more quickly. Because Facebook is not a software company, it does not face the threat of competition in the open-source process, on the contrary, the value of open source in gradually. Facebook users use open source applications can be built quickly, and they are happy to back the code, so that Facebook benefit.
Embrace open source, which means that Facebook must start to write better software. If they know that a software will open since its birth, it must be done properly, increase availability and reliability, because the future people will use it outside. This pressure will bring more value to the house.
Open source has brought opportunity to share challenges. The challenge of open source projects will attract some external excellent staff, and as a result, they also contributed to the company's ability to enhance internal staff. Facebook all day, carrying more than one hundred million people in the communication network, how can do? Only the power of Open Source.
The following is a list of open source projects finishing my existing Facebook more active, welcomed the exchanges and discussions.
Mobile Development Framework: React Native
Data Query Language: GraphQL
GraphQL is Facebook open source data query language. Facebook when building mobile applications, we need to get strong enough with the API data to describe all of the mask, while simple to learn and use, so the development of GraphQL, and support one hundred billion calls per day. Interface GraphQL not like MySQL or Redis this data directly to the interface, but for existing application code. You can GraphQL considered in order to invoke some of the built-RPC method on the application server.
Big Data Query Engine: Presto
Presto Facebook is the development of a distributed SQL engine, is mainly used (from GB to PB) to run the interactive analysis query for various sizes of data sources. The main purpose of Facebook is to create Presto help them analyze data more quickly, because the amount of data Facebook has continued to grow, paced product cycles are becoming faster and faster. Since November 2013 open, subscribers Presto rendering the explosive growth. Such as Airbnb, Jingdong, Dropbox and Netflix and other companies will Presto as their interactive query engine.
PHP execution engine: HHVM
HHVM (HipHop Virtual Machine) open source is Facebook in 2013 PHP execution engine. It uses a JIT (just-in-time) compiler mechanism to achieve high performance while maintaining full support for PHP syntax. HHVM often used as a standalone server, used to replace Apache with mod_php, aimed at implementing the Hack using programs written with PHP. It uses a real-time compilation methods to achieve superior performance, while maintaining the flexibility of PHP developers are used to.
Key-value storage system: RocksDB
RocksDB Facebook is open-source embedded, persistent key-value storage system may be, it is based on Google's LevelDB, but increase the scalability to run on multi-core processors, flash memory can be used effectively to support IO bound, memory, and a write load. Over a period of time, RocksDB very popular in the community, Facebook is its ability to analyze the reasons for the slow query response time due to network delay the elimination of other causes play a role, RocksDB very flexible and can be for a variety of emerging hardware trends custom made. LinkedIn and Yahoo are RocksDB heavy users.
AI hardware platform: Big Sur
In recent years, artificial intelligence and machine learning orientation has made considerable progress. According to Kevin Lee revealed that, Facebook's AI software has been able to read the story, answer questions related to the scene, play games, and through a number of examples to learn unspecified content. As compute-intensive applications, performance is closely related to the size of the data set / hardware performance AI software. Especially in terms of hardware, high-performance microprocessors, memory and graphics processor (Graphics Processing Unit, GPU) for the rapid development of run AI algorithms provide a solid foundation. In order to better serve the large-scale AI computing, Facebook launched a GPU-based, used to train the neural network & ldquo; Big Sur & rdquo; hardware system.
Network simulation testing tools: ATC
Augmented Traffic Control (ATC) to take advantage of Wi-Fi network simulation 2G, 2.5G (Edge), 3G and LTE 4G mobile network environment, test engineers can quickly in a variety of simulated network environment switching, enabling smartphone App and performance in different countries and regions and application environment for testing. ATC is a Facebook internal team developed in 2013 the first Hackathon event tool, the principle actually is the use of Linux traffic control system, calls to netlink API is controlled by pure Python network library pyroute2, and development of its purpose is to ensure more users get the best experience.
Open source database: HydraBase
HydraBase is an upgraded version of HBase database. Facebook HBase is a heavy user, Facebook's HBase database system stores a lot of business-critical data Facebook, including the system of internal control, search indexes, data flow analysis and data fetch and the like. HydraBase HBase stability and higher availability compared to reduce server downtime. HydraBase allows a data field domains across multiple servers, domain server can back up each other, it is possible to greatly reduce the time it takes to restore the data. Facebook Facebook claiming HydraBase annual downtime can be reduced to less than 5 minutes.
Facebook has HydraBase donated to Apache, is currently a lot of code have been incorporated into the HBase.
Relational databases: WebScaleSQL
WebScaleSQL is based on MySQL 5.6 community edition of MySQL adapted general-purpose branch, released under the GPL open source license. WebScaleSQL has done a lot of performance improvements, including: asynchronous client coordination, logic read-ahead, limiting the query, the server thread pool optimization, InnoDB large page support and more. They are very functional WebScaleSQL on & ldquo; Web Scale & rdquo; and ground gas. For example, the thread pool optimization, WebScaleSQL based Mariadb thread pool implementation rewritten and optimized for read and write queue separation redesigned queue priority policy to avoid starvation phenomenon. To know the thread starvation in some scenes is very serious. Especially in the number of concurrent connections are often large Internet applications inside.
Code review tool Phabricator
Code censorship, Facebook open source visualization tools Phabricator. Engineers can easily be on a page for each segment (single-line or multi-line) code for interactive discussion; responsible for reviewing engineers can accept code changes, you can continue to modify the original author questioned the claim, can make their own are not suitable to launch the Code Review ,and many more. Only after the code is clearly accepted in order to be submitted to the Engineer-to-server code base, which is integrated into submission tool for enforcement.
C language event framework: libPhenom
libPhenom event framework is a C language Facebook released for building high performance and scalable system. It supports multi-threading, provides memory management and common data structures, JSON processing. Characteristics are as follows:
C ++ HTTP framework: Proxygen
Proxygen is a Facebook open source support SPDY 3.1 of HTTP framework. The purpose is not to replace Apache, but the ability to create a dedicated, high-performance Web server so that it can be embedded into Facebook to provide Web services existing applications. After Facebook from 2011 to start building a proxy server (Proxygen the name is derived from this), the evolution of the project for several years and tested in a production environment, Facebook will open up its code. Facebook internal benchmarks show done in a Proxygen echo server can support up to 304,197 per second memory based SPDY 3.1 GET requests.
Open Animation Library: Pop
Pop is a Facebook introduced a scalable iOS and OS X Animation Library, reading its news aggregation core technology behind the application Paper is supported by the Pop. In addition to increasing the basic static animation, it also supports Spring and decay dynamic animation can be very easy to build a realistic, physics-based interaction. Pop Animation Library animation is very smooth, because it uses CADisplayLink to refresh the screen (frame), a second refresh frame number 60, close to the game development engine. Pop Anime self-contained, with CoreAnimation systems are very different, but use very similar.
Memcached protocol router: Mcrouter
Mcrouter is based on Memcached protocol router, which is a core component of Facebook cache architecture, at peak times, it can handle per second, 50 billion request. Memcached end customer service will use the standard ASCII encoding Memcached protocol, for the client, Mcrouter like a Memcached server; and for the server side, but like an ordinary Memcached Memcached client. Mcrouter mainly use C ++ to develop and use C library functions developed section, use Ragel has developed a protocol analysis section, use the open source library Folly and Fbthrift handle asynchronous network.
Static code analysis tools: Infer
Infer Facebook development team is in the code submitted an internal review, a static analysis tool used to perform incremental analysis, code submitted to the code base or deployed to find the bug before the user's device. Infer from the OCaml language currently can detect a null pointer access, resource leaks and memory leaks, can C, Java or Objective-C code detection. Facebook Use Infer automatic validation iOS and Android mobile applications on the code, bug reports correct rate of 80%. Infer by capturing the compiler command to compile the files to be converted to be used for the analysis of potential errors intermediate language format. The whole process is the increments, means that only those who have compiled and submitted revised documents will be analyzed Infer. Infer also integrates a large build or compilation tools, including Gradle, Maven, Buck, Xcodebuild, clang, make and javac.
Operating system monitoring tools: osquery
osquery is designed for OSX and Linux operating system detects the frame. The operating system will be exposed as a high-performance relational database that allows users to write SQL queries to view the operating system data. In osquery in, SQL table represents an abstraction like the following:
Loaded kernel modules
Open network connections
Although osquery use a very low-level operating system API, but it allows users to build and use it in Ubuntu, CentOS and Mac OS X on. osquery high performance, small memory footprint, enabling users to execute queries on the entire infrastructure.
Haskell Library: Haxl
Facebook Open Haxl, a concurrent data access and for the efficient development of libraries. This library
On the one hand the use of the traditional advantages of Haskell, for example, express very strong type system, to guarantee the accuracy and security, on the other hand also benefit from GHC (Haskell compiler), high-performance runtime libraries, solve the annoying hidden concurrency data access issues. Haxl simplifies access to remote data, such as a database or web service. A plurality of access requests for the same data source, or simultaneously request data from different data sources, it can batch processing and cache its last results.
Web Application Architecture: Flux
Facebook believes MVC unable to meet their expansion needs, so they decided to use another mode: Flux. Because Facebook is huge code base and large organization, so MVC really quickly become very complex, so they concluded that MVC is not suitable for large-scale applications.
Every time Facebook engineers trying to add a new feature, the complexity of the system into a series of growth, the code becomes & ldquo; fragile and unpredictable & rdquo ;. For developers new to a code base, this is becoming a serious problem. Flux is a Facebook development, the use of application-way data flow architecture implemented for React. Flux application consists of three main parts: scheduler, store and view (React assembly).
Atom-based set of development tools: Nuclide
Android debugging tools: Stetho
Stetho is a debugging tool for Android applications. When Android Application Integration Stetho, developers can access the Chrome, Chrome Developer Tools view in the application layout, network requests, sqlite, preference and so on, all the visualization application (which is more important is not root). Developers can also order it through the tools provided dumpapp line interface to access the application internally.
Android Compilation Tools: Buck
Buck was inspired by Google Blaze, and it was created to deal with complex associated with multiple Android library applications, thereby reducing the build time. After the introduction of Buck, Facebook developed four native Android application using a single tree and build tools that make development easier, smoother, with fewer errors. The initial 38 library through the four applications share a 500 module. After using Buck replace the original Ant-based systems, first run for the tree, build time from 3 minutes and 40 seconds down to 1 minute 30 seconds.
Compared to the traditional Android build tools, Buck With multi-core and parallel technology, which greatly accelerated the speed of compilation Android project. At the same time, several times during compilation, it does not change the module will be marked to incremental compilation ways to further improve the speed. Buck comes compile the script generation, and offers the compilation process unit test code coverage and other data forms, as well as Ant tool can not be compiled module provides a convenient way to compile. Buck with IntelliJ linked closely together, you can generate the IDE project are available through a simple build script, which greatly reduces the cost of development after local IDE to the server migration.
Spring model Java libraries: Rebound
Rebound spring model is a Java library, meeting in Mobile @ Scale posted by Facebook in October 2013, aims to apply the introduction of the real physical world, people feel very close to nature to create animation. Rebound is not a universal physics library, however, the spring model is capable of driving a wide variety of animation. Rebound simple characteristics make it very easy to integrate, as well as building blocks to create such a call, scroll bars, and the switch complex components.
Mobile application interaction design tool: Origami
Prototyping App now more complex, previously used PhotoShop make still pictures can not satisfy a variety of interactive ways to show effects, Paper's chief designer Mike Matas added at the beginning of Facebook recommend that you use Quartz Composer to quickly build application prototypes , while Facebook's design team also received and quickly in love with this tool, in the subsequent application, they encountered a problem: for product designers, Quartz Composer learning curve is too high.
So Mike will lead the development of Origami. December 2013, Facebook open-source-based Quartz Composer plug Origami, Origami designers can quickly build mobile applications interact with the prototype, and then delivered to the engineers to achieve, it is worth noting that no programming background Origami, Paper new release from the project design at the beginning of all the prototypes are based on Origami to achieve.
UI testing tools: huxley
Huxley is a Python-based tool for testing Web application UI, Huxley can record the process of UI operations, and playback automated testing. Automatic testing and benchmarking UI, the UI is not in line with expectations, the UI will save the changes and warn you.
Facebook iOS UI tool: ComponentKit
ComponentKit using functional and declarative (declarative) method to create the interface, and different from the past, using the form ComponentKit unidirectional data flow model is mapped to the variable never immutable component to determine the display view. ComponentKit of declarative looks declarative UI (QML) almost, in fact, not even close. QML UI design is more biased in favor of descriptive, but ComponentKit is good contact between the basic UI and events, so that the event can be separated design and UI design done individually.
iOS Memory monitoring tools: FBMemoryProfiler
FBMemoryProfiler open source is Facebook a tool for analyzing memory usage and to detect iOS circular reference tool library.
Device memory is a shared resource. Improper application may run out of memory, crash, or experienced a significant decrease in performance. When the allocated block of memory, and set up the object, if after using over forgetting to release this memory leak occurs. This means that the system is unable to reclaim memory and handed over to others, which ultimately means that we will gradually run out of memory.
In Facebook, there are a lot of engineers to work on different parts of the code base. This inevitably leaks memory. When the memory leak occurs, engineers need to quickly find and fix them. Some tools may have found a memory leak, but they require a lot of manual intervention. Automation can without the need for more developers situations, faster to find memory leaks. To solve this problem, Facebook made a set of tools to automate the process and fix some of the problems in the code base, this tool is FBMemoryProfiler.