Apache HTTP Server

Happy September! Let's take a look at the activities from the Apache community over the past week: ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.- Next Board ... [More] Meeting: 18 September 2019. Board calendar and minutes http://apache.org/foundation/board/calendar.html ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998 - Countdown to ApacheCon North America and Europe -- we look forward to seeing you in Las Vegas and Berlin -- REGISTER TODAY! https://www.apachecon.com/ - Catch the latest on ApacheCon™,  Q&A With Community Track Leader, Sharan Foga, ASF’S VP of Apache Community Development exclusively on https://feathercast.apache.org/ ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock. - 7M+ weekly checks yield uptime at 99.75%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/ Apache Code Snapshot – this week, 800 Apache contributors changed 1,666,694 lines of code over 3,132 commits. Top 5 contributors, in order, are: Jean-Baptiste Onofré, Stephen Mallette, Andrea Cosentino, Gary Gregory, and Chesnay Schepler. Apache Project Announcements – the latest updates by category. Big Data -- - Apache Myriad 0.4.0 (Incubating) released http://myriad.apache.org Build Management -- - Apache Ant 1.10.7 released http://ant.apache.org Content -- - Apache Jackrabbit 2.16.5 and 2.18.3 released http://jackrabbit.apache.org Databases -- - Apache HBase 2.1.6 released http://hbase.apache.org Libraries -- - Apache Juneau 8.1.0 released http://juneau.apache.org  - Apache Commons Text 1.8 released http://commons.apache.org/proper/commons-text  - Apache Flagon UserALE.js 2.0.2 (Incubating) released http://flagon.incubator.apache.org Servers -- - Apache HttpComponents Core 4.4.12 released http://hc.apache.org Did You Know?  -  Did you know that the ASF's day-to-day operating expenses are offset by the generous donations of our Sponsors? The ASF is supported by 10 Platinum Sponsors, 9 Gold Sponsors, 11 Silver Sponsors, 25 Bronze Sponsors, and 6 Platinum Targeted Sponsors, 5 Gold Targeted Sponsors, 3 Silver Targeted Sponsors, and 10 Bronze Targeted Sponsors https://s.apache.org/w7bw1  - Did you know the following Apache projects are celebrating anniversaries this month? Many Happy Returns to Apache ServiceMix (12 years); Hive, Pig, and Shiro (9 years); Airavata, Bigtop, SIS, and Stanbol (7 years); Curator (6 years); Storm (5 years); Yetus (4 years); DRAT, RocketMQ, and Royale (2 years); and Pulsar (1 year). https://projects.apache.org/committees.html?date  - Did you know that ApacheCon sponsors and community partners include Aiven, AWS, Comcast. CrowdStrike, DataStax, Google Cloud, IBM, HotWax Systems, Instaclustr, Linode, LinuxMagic, Manning Publications, Red Hat, PCCC, ShapeBlue, SK Telecom, StreamNative, The Last Pickle, and more? Learn more at https://www.apachecon.com/ Apache Community Notices:  - Celebrating 20 Years Community-led Development "The Apache Way" https://s.apache.org/ASF20thAnniversary  - ASF Founders look back on 20 Years of the ASF https://blogs.apache.org/foundation/entry/our-founders-look-back-on  - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI  - Foundation Reports and Statements http://www.apache.org/foundation/reports.html  - ApacheCon: Tomorrow's Technology Today since 1998 http://s.apache.org/ApacheCon  - ASF Annual Report for FY2019 https://s.apache.org/FY2019AnnualReport  - The Apache Software Foundation 2018 Vision Statement https://s.apache.org/zqC3  - Foundation Statement –Apache Is Open. https://s.apache.org/PIRA  - "Success at Apache" focuses on the processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache  - Please follow/like/re-tweet the ASF on social media:  @TheASF on Twitter (https://twitter.com/TheASF) and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation  - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/and Twitter account https://twitter.com/ApacheCommunity  - The list of Apache project-related MeetUps can be found at http://events.apache.org/event/meetups.html  - Registration is open for ApacheCon North America 9-12 September 2019 http://apachecon.com/  - Spark + AI Summit 2019 will be held 15-17 October in Amsterdam https://databricks.com/sparkaisummit/  - Registration open for ApacheCon Europe 22-24 October 2019 http://apachecon.com/  - Find out how you can participate with Apache community/projects/activities --opportunities open with Apache Camel, Apache HTTP Server, and more! https://helpwanted.apache.org/  - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby = = = For real-time updates, sign up for Apache-related news by sending mail to [email protected] and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers. [Less]

Farewell, August --we're wrapping up the month with another great week. Here are updates on Apache community's activities: ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.- ... [More] Next Board Meeting: 18 September 2019. Board calendar and minutes http://apache.org/foundation/board/calendar.html ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998 - Countdown to ApacheCon North America and Europe -- we look forward to seeing you in Las Vegas and Berlin -- REGISTER TODAY! https://www.apachecon.com/ - Catch the latest on ApacheCon™,  Q&A With Community Track Leader, Sharan Foga, ASF’S VP of Apache Community Development exclusively on https://feathercast.apache.org/ ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock. - 7M+ weekly checks yield uptime at 99.97%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/ Apache Code Snapshot – this week, 923 Apache contributors changed 13,694,508 lines of code over 3,419 commits. Top 5 contributors, in order, are: Tilman Hausherr, Jun Zhao, Radu Cotescu, Christofer Dutz, and Claus Ibsen. Apache Project Announcements – the latest updates by category. Big Data -- - Apache Flink 1.9.0 released http://flink.apache.org  - Apache NiFi Registry 0.5.0 released http://nifi.apache.org/registry Content -- - Apache Jackrabbit 2.19.4 and Oak 1.8.16 released http://jackrabbit.apache.org Databases -- - Apache IoTDB (Incubating) 0.8.0 released http://iotdb.apache.org Libraries -- - Apache Qpid JMS 0.45.0 released http://qpid.apache.org  - Apache Libcloud 2.6.0 released http://libcloud.apache.org  - Apache ServiceComb 0.5.0 released http://servicecomb.apache.org Did You Know?  - Did you know that the ASF continued guardianship of 190M+ lines of code in the Apache repositories https://s.apache.org/w7bw1  - Did you know that ApacheCon North America is featuring 19 Tracks overseen by Apache Projects and their communities? From Big Data, Community, Content Delivery, Geospatial, Graphs, Integration, IoT, Machine Learning, Mobile, Observability, Streaming, Servers & more --plus Apache project events such as Beam Summit, CloudStack Collaboration Conference, and special workshops-- there's Apache goodness for everyone. Join us! https://www.apachecon.com/acna19/  - Did you know that "Trillions and Trillions Served", the documentary on The Apache Software Foundation, will resume filming at ApacheCon Las Vegas and Berlin https://youtu.be/UvuyBz1qMCE ? Support this project https://s.apache.org/trillions Apache Community Notices:  - Celebrating 20 Years Community-led Development "The Apache Way" https://s.apache.org/ASF20thAnniversary  - ASF Founders look back on 20 Years of the ASF https://blogs.apache.org/foundation/entry/our-founders-look-back-on  - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI  - Foundation Reports and Statements http://www.apache.org/foundation/reports.html  - ApacheCon: Tomorrow's Technology Today since 1998 http://s.apache.org/ApacheCon  - ASF Annual Report for FY2019 https://s.apache.org/FY2019AnnualReport  - The Apache Software Foundation 2018 Vision Statement https://s.apache.org/zqC3  - Foundation Statement –Apache Is Open. https://s.apache.org/PIRA  - "Success at Apache" focuses on the processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache  - Please follow/like/re-tweet the ASF on social media:  @TheASF on Twitter (https://twitter.com/TheASF) and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation  - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/and Twitter account https://twitter.com/ApacheCommunity  - The list of Apache project-related MeetUps can be found at http://events.apache.org/event/meetups.html  - Registration is open for ApacheCon North America 9-12 September 2019 http://apachecon.com/  - Spark + AI Summit 2019 will be held 15-17 October in Amsterdam https://databricks.com/sparkaisummit/  - Registration open for ApacheCon Europe 22-24 October 2019 http://apachecon.com/  - Find out how you can participate with Apache community/projects/activities --opportunities open with Apache Camel, Apache HTTP Server, and more! https://helpwanted.apache.org/  - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby = = = For real-time updates, sign up for Apache-related news by sending mail to [email protected] and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers. [Less]

If Apache NetBeans runs on JDK 8, a range of Ant-based JavaFX sample applications are available in NetBeans to help you get started and learn about JavaFX. However, if NetBeans does not run on JDK 8, the available Ant-based JavaFX samples don't work ... [More] (can't be created) but there's no point in working on fixing that since from JDK 11 onwards JavaFX is no longer part of the JDK and Maven/Gradle-based OpenJFX samples are obvious candidates for integration into NetBeans instead. However, how should that be handled in NetBeans? Before Apache NetBeans 11.1, there was no integration with OpenJFX. Only JavaFX projects and samples were built into NetBeans, which led to a great deal of confusion since when someone sets up an environment from scratch today, they're unlikely to have installed JDK 8. Much more likely, they'll have JDK 11 or 12 and then those JavaFX projects and samples in NetBeans cannot be used, i.e., when you try to create those samples, while running NetBeans on anything other than JDK 8, you're simply told in the wizard that you have the wrong JDK. And then you somehow need to find out that the best thing to do next is use the OpenJFX documentation to set up the OpenJFX samples in NetBeans. That is suboptimal and so Gluon integrated their two sample applications into Apache NetBeans 11.1, i.e., in the most recent release: https://github.com/apache/netbeans/tree/master/javafx/openjfx.samples That is a step forward but still suboptimal, as explained here by Jaroslav Tulach: https://github.com/apache/netbeans/pull/1241#issuecomment-491357016 That entire new module is not needed. Literally, all that needs to be done is that this file needs to be updated with two new template registrations: https://github.com/apache/netbeans/blob/master/java/maven/src/org/netbeans/modules/maven/newproject/MavenWizardIterator.java And, literally, this is all that needs to be added there, since the two OpenJFX samples are on Maven Central and as pointed out above, "NetBeans has a nice support for creating wizards over Maven archetypes." @TemplateRegistration(folder = ArchetypeWizards.TEMPLATE_FOLDER, position = 925, displayName = "#LBL_Maven_FXML_Archetype", iconBase = "org/netbeans/modules/maven/resources/jaricon.png", description = "javafx.html") @Messages("LBL_Maven_FXML_Archetype=FXML JavaFX Maven Archetype") public static WizardDescriptor.InstantiatingIterator> openJFXFML() { return ArchetypeWizards.definedArchetype("org.openjfx", "javafx-archetype-fxml", "0.0.2", null, LBL_Maven_FXML_Archetype()); } @TemplateRegistration(folder = ArchetypeWizards.TEMPLATE_FOLDER, position = 926, displayName = "#LBL_Maven_Simple_Archetype", iconBase = "org/netbeans/modules/maven/resources/jaricon.png", description = "javafx.html") @Messages("LBL_Maven_Simple_Archetype=Simple JavaFX Maven Archetype") public static WizardDescriptor.InstantiatingIterator> openJFXSimple() { return ArchetypeWizards.definedArchetype("org.openjfx", "javafx-archetype-simple", "0.0.2", null, LBL_Maven_Simple_Archetype()); } That literally is all that is needed to be added to the Java source file above, instead of having a completely new module, which doesn't integrate as neatly as the above with the Apache NetBeans infrastructure. (And this is a small tip for anyone else wanting to make their Maven archetypes available to NetBeans: the above is literally all you need to do.) However, the fundamental question remains: how do we notify users of Apache NetBeans that they should be using OpenJFX and not JavaFX? Maybe we should simply remove all JavaFX projects and samples, however that would be unfortunate for anyone using JDK 8. Or maybe the solution is to create a category named "Legacy" in the New Project dialog and then put all JavaFX projects and samples there, so that it's clear that they're not recommended, while still having them available for JDK 8 users? [Less]

To simplify the Welcome Screen and, in particular, replace all links to netbeans.org with netbeans.apache.org, I have created this issue and pull request: https://issues.apache.org/jira/browse/NETBEANS-3020 ... [More] https://github.com/apache/netbeans/pull/1457 All references to netbeans.org are replaced with equivalents at netbeans.apache.org and the News column, which pointed to netbeans.org, is removed from the tab below, while the Blogs column is renamed to News, since newsworthy items now come from here, i.e., from this blog: Also, the Featured Demo on the first tab is removed, best to have as few links to external places as possible, i.e., help reduce potential points of failure, especially here where having that demo in the page doesn't add all that much while removing it reduces the need for external URL calls that could cause problems and slow things down. [Less]

Greetings all. It's time to review the Apache community's activities from the past week: ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.- Next Board Meeting: 18 September ... [More] 2019. Board calendar and minutes http://apache.org/foundation/board/calendar.html ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998 - Countdown to ApacheCon North America and Europe -- we look forward to seeing you in Las Vegas and Berlin -- REGISTER TODAY! https://www.apachecon.com/ - Catch the latest on ApacheCon™,  Q&A With Community Track Leader, Sharan Foga, ASF’S VP of Apache Community Development exclusively on https://feathercast.apache.org/ ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock. - 7M+ weekly checks yield uptime at 99.96%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/ Apache Code Snapshot – this week, 882 Apache contributors changed 1,664,749 lines of code over 3,669 commits. Top 5 contributors, in order, are: Shad Storhaug, Tilman Hausherr, Claus Ibsen, Liang Zhang, and Andrea Cosentino. Apache Project Announcements – the latest updates by category. Content -- - Apache Jackrabbit Oak 1.10.4 released http://jackrabbit.apache.org  - Apache UIMA Java SDK 3.1.0 released http://uima.apache.org Databases -- - Apache HBase 2.0.6 released http://hbase.apache.org - Apache Impala 3.3.0 released http://impala.apache.org/ Libraries -- - Apache Qpid Proton 0.29.0 released http://qpid.apache.org Templating -- - Apache FreeMarker 2.3.29 released http://freemarker.apache.org Servers -- - Apache Tomcat 8.5.45 and 9.0.24 released http://tomcat.apache.org - Apache Traffic Server 8.0.5 and 7.1.8 released http://trafficserver.apache.org Did You Know?  -  Did you know that ASF codebase is conservatively valued at least $20B, using the COCOMO 2 model? https://s.apache.org/w7bw1  - Did you know that workshops at ApacheCon North America will include Apache Beam, Apache OpenWhisk, Apache Spark and Scala, as well as the Apache-licensed Ballerina programming language? Join us! https://www.apachecon.com/acna19/  - Did you know that "Trillions and Trillions Served", the documentary on The Apache Software Foundation, will resume filming at ApacheCon Las Vegas and Berlin https://youtu.be/UvuyBz1qMCE ? Support this project https://s.apache.org/trillions Apache Community Notices:  - Celebrating 20 Years Community-led Development "The Apache Way" https://s.apache.org/ASF20thAnniversary  - ASF Founders look back on 20 Years of the ASF https://blogs.apache.org/foundation/entry/our-founders-look-back-on  - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI  - Foundation Reports and Statements http://www.apache.org/foundation/reports.html  - ApacheCon: Tomorrow's Technology Today since 1998 http://s.apache.org/ApacheCon  - ASF Annual Report for FY2019 https://s.apache.org/FY2019AnnualReport  - The Apache Software Foundation 2018 Vision Statement https://s.apache.org/zqC3  - Foundation Statement –Apache Is Open. https://s.apache.org/PIRA  - "Success at Apache" focuses on the processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache  - Please follow/like/re-tweet the ASF on social media:  @TheASF on Twitter (https://twitter.com/TheASF) and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation  - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/and Twitter account https://twitter.com/ApacheCommunity  - The list of Apache project-related MeetUps can be found at http://events.apache.org/event/meetups.html  - Registration is open for ApacheCon North America 9-12 September 2019 http://apachecon.com/  - Spark + AI Summit 2019 will be held 15-17 October in Amsterdam https://databricks.com/sparkaisummit/  - Registration open for ApacheCon Europe 22-24 October 2019 http://apachecon.com/  - Find out how you can participate with Apache community/projects/activities --opportunities open with Apache Camel, Apache HTTP Server, and more! https://helpwanted.apache.org/  - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby = = = For real-time updates, sign up for Apache-related news by sending mail to [email protected] and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers. [Less]

Hadoop Community Meetup @ Beijing Author: Junping Du (Tencent) & Wangda Tan (Cloudera) Overview On Aug 11th 2019, Hadoop developers/users gathered together at Tencent’s Sigma Center office in Beijing to share their ... [More] latest works, with 12 presentations by engineers from Tencent, Cloudera, Alibaba, Didi, Xiaomi, Meituan, ByteDance (Parent company of TikTok, Toutiao, etc.), JD.com, Huawei. This is also first Hadoop community meetup hosted by Apache Hadoop PMC members. Attendees We received tremendous numbers of participations to the meetup. There’re 200 spots available for registration to attend this meetup in-person, and spots got fully booked in 10 mins. We got 150+ attendees in-person, and 3000+ attendees participated online live sessions. We have participants from dozens of different companies and universities in person, many of them are flying from Shanghai, Hangzhou, Shenzhen and even San Francisco Bay Area! Sessions 1. Hadoop Community Update And Roadmaps Junping Du @ Tencent and Wangda Tan @ Cloudera talked about Hadoop community updates and roadmaps. Junping Du @ Tencent   Wangda Tan @ Cloudera Junping introduced recent trends in the storage field, such as better scalability and moving to cloud. He talked about features like RBF (Router Based Federation), improvements of NameNode scalability, Improvements of cloud connectors and Ozone. Wangda talked about recent trends in the compute field, such as better scalability, moving to clkoud-native environment, containerization works and support of Machine-Learning use cases. He talked about global scheduling framework for better scheduling throughput and placement quality. Recent containerization works in YARN such as runc, interactive docker shell. And YARN-on-cloud initiatives from community such as autoscaling, graceful decommissions, etc. Wangda also talked about Submarine and its release plans. At last, Wangda looked back at releases in 2018/2019, and shared tentative release plan of Hadoop in 2019. Such as 3.1.3, 3.2.1 and what’s new coming to 3.3.0. 2. Ozone: Hadoop native object store Sammi (Yi) Chen @ Tencent talked about native object store project from Hadoop community. Ozone is a strong-consistent distributed object store service. Like HDFS, Ozone has same level of reliability, consistency and usability. It supports S3 interface, so it is not only useful to on-prem big-data workload. It is also a good option to move big data to cloud. Sammi talked about architecture of Ozone, and what’s new in Ozone 0.5 release. 3. YARN 3.x in Alibaba Tao Yang from Alibaba talked about Hadoop use cases in Alibaba. He also talked about how new features in YARN 3.x being used to solve use cases. Tao talked about features like preemption, scheduling, resource over-commitment, scheduling diagnostic, mixed deployment of online/offline workload. Tao also talked about how new features in YARN help to better run Apache Flink on YARN. Tao talked about many interesting features such as MultiNodeLookupPolicy, which can help schedule jobs on a pluggable node sorter. 4. HDFS Best Practices learned from Didi’s production environment. Hui Fei from Didi talked about HDFS best practices learned from Didi’s large scale (hundreds of PBs) production environment. Hui first talked about storage use cases and scale in Didi’s environment. Then Hui talked about functionalities and improvements Didi’s Hadoop team built on top of Hadoop HDFS 2.7.2 such as: Security, NameNode Federation, Balancer, etc. Hui also talked about the status of upgrading production cluster based on Hadoop 2.7.2 to Hadoop 3.2.0. The primary driver of upgrade is to save storage spaces. Didi wants to use features like Erasure Coding in Hadoop 3.x. Didi has upgraded a test cluster (100+ nodes) from 2.7.2 to 3.2.0, has a backup cluster with 2k+ nodes run Hadoop 3.1.1 and will rolling upgrade it to 3.2.0. There’s a primary cluster with 10K+ nodes (with 5 namespaces), will start to upgrade to 3.2.0 starting Oct 5. Submarine: A one-stop, cross-platform machine learning platform Xun Liu @ NetEase and Zhankun Tang @ Cloudera talked about background, existing status and future of Submarine project. Zhankun Tang @ Cloudera   Xun Liu @ Netease Machine learning includes many components like data-preprocessing, feature extraction, model training/serving/management, distributed workload management. Submarine project started by Hadoop community is targeted to achieve these goals by focusing on Notebook experiences. With Submarine, data scientists or machine learning engineer don’t need to understand lower-level platform such as YARN, K8s, Docker container. Zhankun showed a new feature called mini-submarine which allows developers try Submarine locally without installing a YARN cluster. Xun did demos for: ●      Integration of Submarine + Zeppelin notebook. ●      New Submarine web UI to allow data scientists to run jobs and manage models, etc. in the unified user experiences. Xun also talked about companies which are reported using Submarine in production. Such as NetEase, Linkedin, Dahua, Ke.com, JD.com. 6. Hadoop Improvements in Xiaomi Chen Zhang and Kang Zhou from Xiaomi talked about how Hadoop is being used in Xiaomi. They talked about improvements of HDFS’s performance and scalability; Problems/Solutions when trying to platformize YARN. For HDFS side, Chen talked about their improvements of HDFS federation, such as lower the business impact when upgrading single NameNode to federated NameNode. They have also improved NameNode Performance, which now allows supporting 600 millions of objects (files + blocks) in a single NameNode. In YARN, Kang talked about usability improvements in YARN. Such as RMStateStore/History Server, etc. Also, he talked about multi-cluster management tools such as a unified client/RM-UI for multiple clusters. Kang also talked about improvements they have done for scheduling optimization like cache Resource Usage, improvements of utilization and preemption, etc. 7. Key Customizations of YARN @ ByteDance Yakun Li from ByteDance talked customizations of their YARN cluster to handle extra large scale, multi-clusters environment, Including: utilization improvements, stabilization, optimizations for streaming/model-training environment, and multi datacenter issues, etc. For scheduling, Yakun also talked about how they implement Gang Scheduling in YARN, which do scheduling for application instead of node. And it can achieve low-latency, hard/soft constraints. He also talked about implementation of multi-thread version FairScheduler which can push number of container allocation per second up to 3k. In mixed-workloads (Batch, Streaming, ML) deployment part, Yakun talked about they have adopted Docker on YARN support to isolate dependencies. Support CPUSET/NUMA, temporarily skip nodes which have too high physical utilizations, etc. All these efforts can help mixed workload runs well in same cluster. 8. YuniKorn: A New Unified Scheduler for Both YARN and K8s Weiwei Yang and Wangda Tan from Cloudera talked about their works about a new scheduler named YuniKorn (https://github.com/cloudera/yunikorn-core) and how it can benefit both YARN and K8s community. Weiwei Yang (Right) and Wangda Tan (Left) from Cloudera Scheduler of a container orchestration system, such as YARN and Kubernetes, is a critical component that users rely on to plan resources and manage applications. They have different characters to support different workloads. YARN schedulers are optimized for high-throughput, multi-tenant batch workloads. It can scale up to 50k nodes per cluster, and schedule 20k containers per second; On the other side, Kubernetes schedulers are optimized for long-running services, but many features like hierarchical queues, fairness resource sharing, and preemption etc, are either missing or not mature enough at this point of time. However, underneath they are responsible for one same job: the decision maker for resource allocations. They mentioned the need to run services on YARN as well as run jobs on Kubernetes. This motivates them to create a universal scheduler which can work for both YARN and Kubernetes and configured in the same way. In this talk, Weiwei and Wangda talked about their efforts of design and implement the universal scheduler. They have integrated it with to Kubernetes already and YARN integration is working-in-progress. This scheduler brings long-wanted features such as hierarchical queues, fairness between users/jobs/queues, preemption to Kubernetes; and it brings service scheduling enhancements to YARN. Most importantly, it provides the opportunity to let YARN and Kubernetes share the same user experience on scheduling big data workloads. And any improvements of this universal scheduler can benefit both Kubernetes and YARN community. 9. HDFS cluster improvements and optimization practices in Meituan Dianping Xiaoqiao He from Meituan Dianping talked about Hadoop cluster scalabilities now. Their Hadoop cluster keep growing since 2015. By far, there’re more than 30k nodes in the Hadoop clusters. He shared many details and practice about the infrastructure of physical deployments, especially on solution for cluster across multiple regions. In the last part, Xiaoqiao shows some practices for optimizing HDFS cluster, such as: improve the Namenode restart process and rebalance for Namenode workload, etc. 10. Evolution of YARN in JD.com Wanqiang Ji from JD.com talked about how YARN evolves to support JD.com’s business needs. In the last 3 years, maximum number of nodes in a single YARN cluster scales from 3k, 5k, 10k to 16k. Internally there’re works to balance resources between YARN/K8s cluster. Also there are improvements of container eviction policies to make sure nodes won’t crash or restart when machine’s physical utilization grows above a certain level. 11. Lessons learned from large scale YARN cluster operation @ Tencent Jun Gong and Dongdong Chen from Tencent talked about their works to support large scale YARN cluster inside tencent. Gong Jun @ Tencent Dongdong Chen @ Tencent Jun and Dongdong shared inside Tencent, they widely used SLS to figure out bottleneck of scheduler, many of the scheduler improvements have contributed back to the community. After optimization, in their production cluster, they have 2k+ queues, 8K+ nodes, 5k+ concurrent jobs. And they can achieve 3k+ container allocations per second, and more than 100 millions container allocations per day. Also, Jun and Dongdong shared how they uses YARN CGroups parameters to fine-tune CPU/Memory/Network shares for launched YARN containers in a multi-tenant cluster. 12. Run Spark and Hadoop on ARM Rui Chen and Sheng Liu from Huawei shared their works to run Spark and Hadoop on ARM. Rui Chen   Sheng Liu   Rui and Sheng shared the motivation of running hadoop and spark on ARM platform which is for high performance and power efficiency. After that, they went ahead to share status of ARM support for hadoop and spark and details of building release Tarball on ARM platform include parameters, and issues. In the last part, they introduced how hadoop/spark release work can make sure proper testing for arm platform and they were building a community called OpenLab to make sure the process more smoothly.   Acknowledges Thanks everyone for contributing this successful event in one way or another, such as following speakers: Sammi Chen, Jun Gong and Dongdong Chen from Tencent, Weiwei Yang, Zhankun Tang from Cloudera, Wanqiang Ji from Jingdong, Tao Yang from Alibaba, Chen Zhang and Kang Zhou from Xiaomi, Hui Fei from Didi, Rui Chen and Sheng Liu from Huawei, Xiaoqiao He from Meituan Dianping , Yakun Li from ByteDance, and Xun Liu from Netease. And especially thanks Chunyu Wang, Summer Xia, Katty Ma for organizing the meetup! [Less]

Below is a scenario by Jan Lahoda, the creator of LSP integration for Apache NetBeans, for how to integrate the bash language server with Apache NetBeans, including syntax highlighting. Setting Up Install npm (and node.js). On Ubuntu, e.g., do ... [More] "apt install npm", though something different will be needed on Mac OS X. Create a directory in which we are going to work, have a terminal opened in that directory. Install the bash-language-server: npm install bash-language-server On Mac OSX: npm install bash-language-server --unsafe-perm This will install the server into the current directory. Try the bash server: ./node_modules/bash-language-server/bin/main.js --help You should see something like this: Usage: bash-language-server start bash-language-server -h | --help bash-language-server -v | --version Create a NetBeans module. Create a File Type (Module Development/File Type), mime type: text/sh, file extension: sh Syntax Coloring via TextMate Download the TextMate grammar file here, and put it alongside the newly created DataObject: https://raw.githubusercontent.com/microsoft/vscode/master/extensions/shellscript/syntaxes/shell-unix-bash.tmLanguage.json Add "TextMate Lexer" as a dependency of the module. Into the DataObject add this annotation: @GrammarRegistration(grammar="shell-unix-bash.tmLanguage.json", mimeType="text/sh") GrammarRegistration is: import org.netbeans.modules.textmate.lexer.api.GrammarRegistration; This should lead to syntax highlighted source for .sh bash files taken from the TextMate grammar file. Language Support via the Language Server Next, we need to add language support using the language server. Add "LSP Client" and "MIME Lookup API" as dependencies of the module. Create a new class, ShellClient, in the module, put this into it, (replacing "" with the absolute path to "node_modules/bash-language-server"): import java.io.IOException; import org.netbeans.api.editor.mimelookup.MimeRegistration; import org.netbeans.modules.lsp.client.spi.LanguageServerProvider; import org.openide.util.Exceptions; import org.openide.util.Lookup; @MimeRegistration(mimeType="text/sh", service=LanguageServerProvider.class) public class ShellClient implements LanguageServerProvider { @Override public LanguageServerDescription startServer(Lookup lkp) { try { Process p = new ProcessBuilder("/bin/main.js", "start").start(); return LanguageServerDescription.create(p.getInputStream(), p.getOutputStream(), p); } catch (IOException ex) { Exceptions.printStackTrace(ex); return null; } } } You may need to explicitly call node in the above code, i.e., as follows: Process p = new ProcessBuilder( "/usr/local/bin/node", "/bin/main.js", "start").start(); Build and start the module. Caveat: the language server is started only for files that are inside a project, so create (any) new project, and inside the project, put a shell file. E.g. copy "bin/netbeans" as "test.sh" into the project. Open it in the editor - there should be syntax highlighting, Navigator, and code completion should show something, etc. [Less]

Sharan Foga, Apache Software Foundation (ASF)’s Vice President of Apache Community Development, will lead the Community Track at ApacheCon North America, 9-12 September, in Las Vegas. In our Q&A below, Sharan highlights what to ... [More] expect from the Community Track and her role at Apache. You can listen to Sharan’s Apache Feathercast podcasts here: Part 1:  ApacheCon North America: and Part 2: ApacheCon Europe. Q: Introduce yourself. What are you working on at ApacheCon this year? A: I've been involved with Apache for over a decade. I’m the Chair of the Community Development community. I help organize ApacheCon tracks and content. Apache is built on communities, and the ApacheCon Community Track enables us to make sure that we encourage the health, diversity, and longevity of all our communities. This year we'll have 24 talks in just over three days at ApacheCon North America. Q: Tell us about the Community Track. What does it offer new attendees? A: Each year, we design the ApacheCon Community Track to cater to a diverse set of audiences. For new attendees, the track provides a great introduction to the “Apache Way”, highlighting everything about our culture, including how we work, interact and communicate. The track also provides an overview of how projects become part of the ASF’s Apache Incubator program, from submitting a proposal to how to graduate to a top-level project. We also have a great lineup featuring some of the ASF founders at the Founders Panel. For example, Jim Jagielski is going to be speaking about the “Apache Way”. Lars Eilebrecht will present Behind the Scenes of the Apache Software Foundation. Another talk stood out to me: it's Ruth Holloway on empathy and compassion, trying to get people to imagine themselves in somebody else's shoes effectively. Her talk is titled A Mile in My Shoes: How Empathy & Compassion Can Transform Your Work & Your Life. We also have some speakers from the Chinese community in our track at ApacheCon North America. I'm also looking forward to some of the behind-the-scenes talks -- to get perspective from some of the founders on how Apache and the ASF have changed in the 20 years that the ASF has been going; where they started all those years ago and where we are now.  Q: What value does the track have for people who are not new to ApacheCon? A: One area that is quite interesting within open-source is that to have a more balanced community, people make a lot of contributions that are not related to coding. We have contributors who create art, manage the writing, or even handle interviews. These non-coding activities contribute to the health and activity of the community. So one of the things we talk about in our community track deals with understanding the people that are not coders - not developers - but do add some real value to your community.  Q: What benefits do employers get from sending their employees to this track? A: A lot of employers use open-source software. If a corporation is relying on something that is open-source, it makes sense to be involved in the communities that create that open-source material. Doing this provides assurance that you have input on how the project goes forward, besides being actually involved in contributing. Participating helps safeguard your reliance on the open-source software you're using, and coming to ApacheCon gives your employees an understanding of how Apache works because not all open source foundations work in the same way.  Q: One more question: I know you are involved with ApacheCon Europe Track. How does it differ from ApacheCon North America Community Track? A: ApacheCon Europe is going to be in Berlin from the 22nd to the 24th of October. We have two days of the community track and some of the topics are similar -- we have the same theme around Apache, the Apache way, Incubator. We also have the Podling's Shark Tank. One thing that's a bit different for the EU is that there will be talks around Apache governance and the board members. For example, a new board director will share their story about their first six months on the ASF board. We often hear about being an Apache contributor, developer, or a committer but it is not often you get to hear about what's it like to be a board member. This talk should be quite interesting for attendees because anyone can eventually become an Apache board member thanks to Apache flat organizational model.  Another ApacheCon Europe Community Track related to the board is about quarterly reports. All the Apache projects need to report to the board every quarter, which can make for hundreds of board reports. This talk covers how board members handle reading all those project reports that are submitted. And one of the highlights that we've got in the ApacheCon Europe Community Track is about the concept of a community but taken to a different level, looking at something akin to shared neurons. We at Apache often talk about communities as one giant brain, and out of that collaboration, we get a lot of great ideas and a ton of creativity. There are some quite interesting things that we're running in the ApacheCon Europe Community Track that are not on the schedule for North America. If you get the chance to attend both, that would be great.  ApacheCon Q&As and Feathercasts  * ApacheCon Q&A and Feathercast with Rich Bowen * Big Data Track Q&A and Feathercast with Felix Cheung * Streaming Track Q&A and Feathercast with Felix Cheung * Machine Learning Track Q&A and Feathercast with Felix Cheung * Geospatial Software Track Q&A and Feathercast with Geoge Percivall * Keynote and Hackathon Q&A and Feathercast with Samaira Mehta Important Dates and Deadlines * Register at https://apachecon.com/acna19/. * Discounted Hotel Rooms: Book your room before prices go up 29 August! We are offering special room rates at the Flamingo Hotel and Casino. Secure your spot by registering at https://www.apachecon.com/acna19/location.html. Ways to Get Involved, or Get Questions Answered * Join our mailing lists: http://events.apache.org/involved/mailing-lists.html * Email the ApacheCon planners directly: [email protected] * Follow us on Twitter: @apachecon * Join us on Slack * Visit the ApacheCon website [Less]

Happy Friday! We've had a great week. Let's review what we've been up to: ASF Annual Report – The Apache® Software Foundation announces Annual Report for 2019 Fiscal Year - Press release https://s.apache.org/w7bw1 - Full ... [More] report https://s.apache.org/FY2019AnnualReport ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws. - Next Board Meeting: 21 August 2019. Board calendar and minutes http://apache.org/foundation/board/calendar.html ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998 - Countdown to ApacheCon North America and Europe -- we look forward to seeing you in Las Vegas and Berlin-- REGISTER TODAY! https://www.apachecon.com/ - Catch the latest on ApacheCon™ Big Data, Machine Learning and Streaming Tracks with Track Lead Felix Cheung exclusively on https://feathercast.apache.org/ ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock. - 7M+ weekly checks yield uptime at 99.96%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/ Apache Code Snapshot – this week, 840 Apache contributors changed 13,780,155 lines of code over 3,721 commits. Top 5 contributors, in order, are: Etienne Chauchot, Mark Robert Miller, Mark Thomas, Tilman Hausherr, and Henrik Krohns. Apache Project Announcements – the latest updates by category. Content -- - Apache UIMA 2.10.4 released http://uima.apache.org/ Database -- - Apache Druid (Incubating) 0.15.1 release http://druid.incubator.apache.orgHTTP -- - Apache HTTP Server 2.4.41 released http://httpd.apache.org/ - Apache Traffic Server 8.0.4 and 7.1.7 are released http://trafficserver.apache.org/ Libraries -- - Apache Log4j 2.12.1 released http://logging.apache.org/ - Apache Commons BeanUtils 1.9.4 released http://commons.apache.org/proper/commons-beanutils/ - Apache Commons VFS 2.4.1 released http://commons.apache.org/proper/commons-vfs/ Network-Server -- - Apache Qpid Proton-J 0.33.2 released http://qpid.apache.org Did You Know?  - Did you know that at this year's ApacheCon Europe/Berlin include Miguel Gamiño, Executive Vice President Global Cities at Mastercard, will discuss new models of urban collaboration in response to shared challenges? Learn more at https://aceu19.apachecon.com/session/city-possible-addressing-shared-urban-challenges-harnessing-super-power-collaboration  - Did you know ApacheCon North America keynoters include sci-fi author David Brin; IBM Fellow Chris Ferris; "Father of Java" James Gosling; 11-year-old CoderBunnyz CEO Samaira Mehta; the ASF Founders Panel; DataStax co-Founder Jonathan Ellis; plus ASF President Sam Ruby on the State of the Feather, the Apache Community Development panel, and more? https://www.apachecon.com/acna19/s/#/schedule  - Did you know that 202 Top-Level communities are overseeing 332 Apache projects and sub-projects? https://projects.apache.org/ Apache Community Notices:  - Celebrating 20 Years Community-led Development "The Apache Way" https://s.apache.org/ASF20thAnniversary  - ASF Founders look back on 20 Years of the ASF https://blogs.apache.org/foundation/entry/our-founders-look-back-on  - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI  - Foundation Reports and Statements http://www.apache.org/foundation/reports.html  - ApacheCon: Tomorrow's Technology Today since 1998 http://s.apache.org/ApacheCon  - ASF Annual Report for FY2019 https://s.apache.org/FY2019AnnualReport  - The Apache Software Foundation 2018 Vision Statement https://s.apache.org/zqC3  - Foundation Statement –Apache Is Open. https://s.apache.org/PIRA  - "Success at Apache" focuses on the processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache  - Please follow/like/re-tweet the ASF on social media: @TheASF on Twitter and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation  - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/and Twitter account https://twitter.com/ApacheCommunity  - The list of Apache project-related MeetUps can be found at http://events.apache.org/event/meetups.html  - Registration is open for ApacheCon North America 9-12 September 2019 http://apachecon.com/  - Spark + AI Summit 2019 will be held 15-17 October in Amsterdam https://databricks.com/sparkaisummit/  - Registration open for ApacheCon Europe 22-24 October 2019 http://apachecon.com/  - Find out how you can participate with Apache community/projects/activities --opportunities open with Apache Camel, Apache HTTP Server, and more! https://helpwanted.apache.org/  - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby = = = For real-time updates, sign up for Apache-related news by sending mail to [email protected] and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers. [Less]

CEO and Founder of CoderBunnyz, CoderMindz At ApacheCon North America keynote speaker Samaira Mehta, CEO and founder of CoderBunnyz and CoderMindz her keynote, “Yes, You Could Help Build the Next Disruptor!!”, as well as the ... [More] Hackathon, “Yes, 1 Billion Kids Can Code”. You can listen to Samaira’s full interview on our Apache Feathercast podcast here: https://feathercast.apache.org/2019/08/15/apachecon-qa-with-keynote-speaker-samaira-mehta-ceo-and-founder-of-coderbunnyz-codermindz/. To learn more about Samaira’s keynote and hackathon and ApacheCon North America, see the links below. Q: Tell us about yourself and what you do. A: I'm 11 years old and I am the CEO and founder of CoderBunnyz and CoderMindz, both board games intended to teach kids the concepts of computer programming and artificial intelligence in a fun, exciting way. I've started an initiative called “Yes, One Billion Kids Can Code” with the goal of making coding and stem tools accessible in every corner of the world. To achieve this goal, I have been doing workshops such as the hackathon that I will be doing at ApacheCon as well as connecting with companies and organizations in every corner of the world.  Q: What can attendees look forward to at your keynote and the hackathon? A: I'll be giving my keynote on September 11 at 9 a.m. where I'm focusing on how everyone can nurture the next disruptor. I will be using interactive ways to prove that we have all grown very accustomed to technologies, but I like to try to remind people that technology is not the future - we people are.  That same day, at 11 a.m., I will be doing a CoderBunnyz workshop and hackathon. It is a “Yes, One Billion Kids Can Code” workshop and hackathon. It's one of the workshops targeted towards getting my one billion kids initiative, making it all possible. Q: What key takeaways can attendees learn from your presentation? A: My main goal for my talk is to prove that everyone can create and build the next disruptor. I will talk about the superpowers that we have and how kids need the guidance and support of the ApacheCon audience. My goal is to encourage them to help create the next disrupter, help create the next child who creates something big, and be their mentor to help them follow their passions and dreams. I’m excited for both the keynote and the hackathon. I'm looking forward to meeting so many amazing bright minds and learning from them as well as seeing how they can help me with my Billion Kids mission. ApacheCon Q&As and Feathercasts  * ApacheCon Q&A and Feathercast with Rich Bowen * Big Data Track Q&A and Feathercast with Felix Cheung * Streaming Track Q&A and Feathercast with Felix Cheung * Machine Learning Track Q&A and Feathercast with Felix Cheung * Geospatial Software Track Q&A and Feathercast with Geoge Percivall Important Dates and Deadlines * Register at https://apachecon.com/acna19/. * Discounted Hotel Rooms: Book your room before prices go up 29 August! We are offering special room rates at the Flamingo Hotel and Casino. Secure your spot by registering at https://www.apachecon.com/acna19/location.html. Ways to Get Involved, or Get Questions Answered * Join our mailing lists: http://events.apache.org/involved/mailing-lists.html * Email the ApacheCon planners directly: [email protected] * Follow us on Twitter: @apachecon * Join us on Slack * Visit the ApacheCon website [Less]